I made my first crawler with crawler, or node-crawler as it might be known on github. In this post I will just be briefly covering how to get started with it, as the source code of my crawler is a little buggy at the moment and I don’t want to go over it here just yet.

There are many ways to go about getting started with a crawler using nodejs, but when deploying said crawler one might run into all kinds of rabbit holes when it comes to issues relating to making large volumes of requests over a shot period of time. In any case this is just a getting stared post with crawlers using nodejs, and some external libraries for making the process go a little quicker. It might be better to take a completely different approach to this sort of things depending on the long term goals of a project, but if you are just looking for some kind of starting point what I have written here might be of value.

Read More

In lodash there is a method that can be used to break a collection into two groups one of which meets a condition that is given in a function that is passed to it, and another that does not meet that condition. This is of course the _.partition method. Te return value is an array of arrays where the first element is all the elements that meet a given condition and the second element is all elements that do not meet the given condition.

When it comes to the partition method in lodash, it is also not to hard to work out some simple solutions for doing more or less the same with plain old vanilla javaScript. So then lets look at some examples of splitting a collection into two parts with lodash, as well as a few native javaScript solutions for this that do not require lodash.

Read More

Time for yet another one of my posts on lodash, today I will be writing about the _.includes method, and why It might be useful in some situations when working on a project where lodash is part of the stack.

The lodash _.includes method is one of the collection methods in lodash that will work with Arrays, and Objects in general, and even strings. The nature of the lodash includes method is that it can be used as a way to test if a value is included in a collection or not. There are many other lodash methods as well as native javaScript solutions for doing the same thing as what the lodash includes method does. So with that said the lodash includes method might not be one of the methods in lodash that helps to build the most convincing case to use the full lodash utility library in a project.

Still there is what the lodash includes methods does, and there are all the other ways of doing the same thing both with and without lodash, so I thought I would take a moment to write a post around this topic for what it is worth.

Read More

I have been writing about lodash a lot these days, I feel that it is something that is still worth covering at least at the time that I first wrote this post anyway. It is true that many of the methods are now native in the late javaScript specs, but there are of course methods that are not. In addition it is true that many of the methods in lodash work a little differently compared to any native javaScript counterpart. This appears to be the case with _.forEach and the native Array.prototype.ForEach method. As they will both do the same thing, but with some significant note worth differences.

For one thing the lodash for each method is one of the many collection methods of lodash, which means that the method will work with collections in general, not just arrays. Also there are a few more little differences such as what happens when something is returned in the body of a function that is given to lodash for each which can be used as a way to stop looping early.

When it comes to the lodash forEach method and the native javaScript array prototype counterpart it seems that a lot of developers have this negative or positive attitude with the use of such methods. It is a pattern that I have observed in blog posts such as this and from discussions over and over again. My attitude with the use of lodash forEach or naive forEach is that the use of it is fine when it comes to quick code examples now and then. However it certainly goes without saying that it is not the only tool in the toolbox when it comes to looping over the contents of an array, or object in general for that matter. It is important to be aware of the other options in lodash, and in javaScript in general, and realize that sometimes it is best to go with some other option.

Read More

These days I have been playing around more with a node.js project I am familiar with called marked. This is a package that can be used to parse markdown into html. In addition to the usual use case of parsing to html, it is possible to define a custom renderer that can be used to render out plain text, or some other standard other than html.

In any case it is a great little package when it comes to working with markdown source, so it’s worth a post for sure. So lets look as some examples of this one.

Read More