Node.js and io.js - Very different in performance

tl;dr - Depending on whether you use Node.js or io.js, you may experience performance differ by a factor of more than 5 for identical code.
A blog post from geekregator.com, if you wanna to read it in Vietnamese, please wait its comming :))

First of all, let me make something clear. This is not, and cannot be, a comprehensive test. Every application is different. Depending on what your Node application does, my findings may or may not apply to your use-case.

What I tested

To test JavaScript I used Node and chose three different ways of implementing this algorithm, using a regular array, typed-array or buffer. Today I want to use the same benchmark to show how much performance can vary between Node.js and io.js. For those of you who don't know it yet, io.js is a fork of Node.

I will use the same code as in my previous article. You can find it at the end of that article.

Node and io.js have one important thing in common. They both rely on Chrome's V8 JavaScript Engine. But they use different versions of V8. So when I'm comparing Node and io.js, I'm actually comparing the different versions of Chrome's V8 JavaScript Engine they use.

For my previous test I used Node 0.10.32 as that is the one that comes with OpenSuse 13.2, the Linux distribution that I'm using. But for today's test I will use Node 0.10.35 and io.js 1.0.2 as these are the latest available releases. And yes, I am aware that io.js describes that release as "unstable". But it's all I've got. :)

Just as I did for my previous article, I ran each test 7 times and then used the median time as my result. I was running the tests on an Intel i7-4771 3,5GHz using a 64-bit OpenSuse 13.2.

Here are the results:
Node.js 0.10.35 io.js 1.0.2
Buffer 4.259 5.006
Typed-Array 4.944 11.555
Regular Array 40.416 7.359
(Times are listed in seconds)


As you can see, the performance for typed-array and regular array varies quite dramatically.

Using a buffer, io.js takes almost 18% longer than Node. Not nice to see, but not something to really worry about.

The typed-array test takes more than twice as long with io.js than with Node. Now that is a pretty significant difference.

But with regular arrays it is the other way around, and even more pronounced. Here Node takes more than 5x as long as io.js.

I must say that I'm quite disturbed by how much performance can differ. For such a mature product, the V8 JavaScript Engine shows way too much variation in performance between versions. I could live with that if performance would always increase with each version. But the data clearly shows that it can go either way. And that should not happen.

Conclusion

There is no clear winner. Sometimes Node is faster and sometimes io.js is. Also keep in mind that this test is far from anything you would do in real-life.

Even different versions of Node can behave differently in my experience. I remember that there was a regression with a development-version of Node which caused slow performance with buffers. See this issue on GitHub.

This can be extremely important if you have a project with heavy CPU-use and which scales to several servers on something like AWS or Azure. You could potentially be wasting a lot of money if your application performs poorly with the version of Node/io.js that you use. You could be needing more servers than necessary because of that.

If you take only one piece of information from this article, then let it be to always remember to test how your code performs with different versions of Node/io.js.

Related