Some Ways to walk a file system in node.js
As I work to expand my content on node.js, I have come around to working out some examples on how to walk a files system. This includes both my own vanilla js solutions, as well as some walkers that other people have made for node.js, such as klaw, and node-dir, just to name a few. In this post I will be covering some options, and if you are looking into this sort of thing for your own project hopefully you will find this post helpful.
1 - What to know before hand.
This is a post on some node.js options for walking a file system, or in other words looping over the contents of a path recursively to preform file io operations on a whole bunch of content. There are many projects that do this in one way or another, or porjects that make use of a file walker of some kind or another like rimraf.
2 - Lets look at the options with file system walking in node.js
So I will be covering methods that are available in the built in node.js modules that can be used to make your own file sysstem walker from the ground up, along with code examples. I will also be covering some npm packages as well, and linking to other relevant posts on this if you want to read more about each of them.
2.1 - fs.readdir: As a vanilla js node.js solution for walking a file system
The fs.readdir method is one of many useful methods in the node.js built in file system module. When I call it I give it a directory, and a callback, and I can do something with the array of items that are in that are in that given directory. This method by itself is not a file system walker, but this method used with several other methods, inside a method that is called recursively can result in a simple walker.
If you start playing with this option you might end up with something like this:
A more advanced example might introduce use of promises, and breaking things down into more methods along with an events that can be used to define what to do for each item instead of just logging the path to the console. This might be a crude example, but it works, and I can work from here into something not so crude.
If interested in learning more about this, you might want to read my full post on using fs.readdir as a file system walker solution.
2.2 - Using klaw, and through2
klaw is a descent option that I have come to like a lot, and it can also be used with the popular through2 project that helps with node.js object streams.
For more on klaw I have a post on klaw in which I get into this one in further detail.
2.3 - walk
So the npm package that is just simple called walk, is also a decent solution for just quickly getting this done, and moving on with a project.
In addition to the onFile method there are many other such methods for directories, and error handling as well. For more on walk you might want to check out my post on walk for more.
2.4 - node-dir
So node-dir is an option that I have used in the past with some projects, although I do not use it much these days. It is an older project, and it looks like it is no longer supported, also it does not have many of the features that I like it in walker.
I wrote a post on node-dir as well a while back.
2.5 - Using glob
Glob is another npm package that can be used to match files using a glob pattern, another useful tool for making a file system walker.
3 - Conclusion
There are many more walkers out there, also there are other ways to make a walker with node.js by itself as well when it comes to making a walker from the ground up. Many of these projects make use of streams. As such I might update this post as I write more content on this subject.