What is Node.js used for?

As you can guess, there must be a reason why the tech giants have begun using this open source, community-loved engine. The most important one is that the Node.js runtime is built on the popular Chrome V8 JavaScript engine. In an era of JavaScript front-end framework applications, it allows us to use the same programming language for both front and back-end.

So, what Node.js is used for? Usually, it’s for server-side web applications and web development. But that’s not all. Being modular, highly scalable and accessible via the CLI (Command Line Interface) on each platform gives Node.js its various use cases.

A little history of Node.js, JS Node and JavaScript

The JavaScript programming language was developed to execute in the browser environment and is arguably one of the most popular programming languages for development of event driven web applications enabling a rich user experience. Event driven web applications are those that are required to respond to events or actions generated within a system or by users of the system. Its popularity stems from the majority of web browsers incorporating a dedicated JavaScript engine to interpret the programming language. As the name implies, it is an open source scripting language loosely inspired by the Java programming language. It was developed to allow the enhancement of HTML web pages with animated effects, responding to user input and interactions in real-time where fast event driven processing is paramount. JavaScript engines can now be found embedded in other types of host server software including databases, office applications and runtime environments for the application development.

What is Node.Js, exactly?

In short, Node.js is a multi-platform environment. It was created to allow developers write web-server applications in JavaScript.

The history of the web server-side implementation of JavaScript took off with Microsoft’s Internet Information Services (IIS) web server followed by a number of open source server-side JavaScript implementations including Node.js / JS Node. Note that Node is a server installed application that interprets JavaScript files which have the .js extension; hence Node.js and JS Node are synonymous terms.

You might be also interested in the article:

Tech

10 Steps to Becoming a JavaScript Developer

An event driven web application running on the Chrome V8 JavaScript engine also offers fast execution times compared to other available programming language options for web development. Node.js enables JavaScript to be applied outside of the browser environment as command line executables through to the HTTP server that underpin the website. A server needs to concurrently send and receive data to and from multiple devices. Node.js is used to facilitate this by making development of asynchronous programming easy and convenient. Node.js is an open source software, and its installation on a server provides the Node program which itself enables JavaScript files to be executed on that server. Node.js is also cross-platform software, meaning that it is operating system independent such that it works on Linux, Windows and macOS.

Using JavaScript programming for both the browser and on the web server-side has productivity benefits in the application development process by only requiring one set of programming skills whilst also simplifying the implementation of data exchange between the client and server environments with the availability of common functionality.

The key question is, how is it possible? There are many factors affecting Node’s ability to perform in this way.

Node.js is single threaded and runs asynchronously in one event loop. The secret lies in using the power of non-blocking I/O architecture. This means that you should use asynchronous code (running in the background) as much as possible. This incredible real-time processing power can be used with data streams for building scalable network applications managing multiple concurrent connections using the simplified event driven JavaScript runtime that Node.js provides. Blocking occurs when the execution of a Node.js process is forced to halt until a separate operation completes. This is a feature of synchronous processing, for example in Node.js all file system operations have synchronous and asynchronous forms. A synchronous file system read operation will halt Node.js thread execution until the file system read is complete. By contrast an asynchronous file system read operation will allow non-blocking Node.js thread execution to continue whilst the function waiting for the file system read to complete waits for the file system read complete event to be flagged.

Node.js is a single threaded event driven platform, that is everything that happens is as a single sequence of actions and in response to events occurring in real-time. Operations undertaken in Node.js are handled using the libuv library which provides the event loop mechanism. The event loop is part of the same single threaded function as the asynchronous code. Users are disassociated from event loop itself, Node.js automatically enters the event loop and only exists once all required callbacks have been completed in common with the operation of JavaScript web applications.

An additional development benefit is that Node.js does not require direct interactions with I/O leading to no requirement for locks, eliminating the possibility of processes becoming dead locked through the use of asynchronous versions of Node.js standard library methods. Dead locking occurs when synchronous events cause blocking of functions whilst waiting for an event to occur. Whilst blocking is not an issue for a purely synchronous single threaded application, a non-blocking application may not be the best solution, particularly for CPU intensive tasks. Where a synchronous multi-threaded Node.js application is required then managing blocking is a challenge that would need to be carefully considered.

Node.js and multiple cores

Although Node.js is designed to be single threaded, it has the facilities to operate across multiple cores through the decomposition of the thread into child processes run across the cores using shared sockets. This enables development of load balanced processes. A cluster module provided by Node.js allows each child processes to maintain their own single threaded event loop in a transparent manner to the user.

Perfect bidirectional real-time data (also audio and video) streaming is often the strongest reason as to why Node.js is used instead of other technologies. Single-threading is not an issue (Node is able to run statelessly in a cluster using, e.g. the PM2 library) but unfortunately, it doesn’t handle big data processing well.

Let’s look at the most popular use cases for Node.js.

Event loop diagram - the heart of the Node.js V8 engine.

What Node.js is used for - when working with Microservices

This choice of name wasn’t a random decision. The full potential of its architecture is utilized by building web applications as small nodes (parts). Does that sound familiar? Microservices are applications that need to be lightweight, blazing fast, scalable and stateless. Deploying thousands of small (but powerful) nodes in clusters and daemon is available mostly out-of-the-box with tools like PM2.

You might be also interested in the article:

Tech

How to use JavaScript Classes? Three Different Ways

Well, Node.js is light and tiny, but what about connecting to other services? Thanks to the npm package manager, you can use all of its content, including web servers, databases connectors, REST & GraphQL implementations and instant integrations with various API providers, such as AWS. Note that open source alternatives to the npm package manager are available.

Web building has probably never been so easy as it is with the express framework. Nowadays, a complete web server could be written in a couple of lines. Node.js is also intended to be fast also in use - ready applications can be deployed to a zero-configuration Heroku server with a line of CLI command.

Breaking down an application into smaller ones also overlaps with Kubernetes, the service that facilitates automation and configuration of containerized applications. You can compare it to Lego blocks. With a decent amount of small bricks, you can create complex, effective and stable results. Due to the nature of their simplicity, Node.js web applications offer the high availability needed in production environments. You can also accelerate Node.js troubleshooting by aggregating logs from any application or service, searching across log in real-time, and live tail.

The simple architecture of an application built in Node.js:

Real-time communication in Node.js

Building a low-latency live server with opened bidirectional connections is a weak point of many technologies. In some of them, e.g. PHP which runs per request and not as a still opened socket, it’s just really hard to achieve due to the architecture.

Of course, it’s possible to build it in other programming languages (e.g. Python, Java), but building would still require a lot of development effort and workload. Then comes the time for implementing the connection on the front-end. Fighting with discrepancies of encoding, modes, and protocols are not something we like to spend time on.

Is there another, developer-friendly solution?

Of course, there is. One of the great things about Node.js is its scalability. When you don’t need to have tons of individual settings, just download the npm package manager and use it with a few lines of code. It’s the same with sockets. In this case, let’s use socket.io as an example. It’s an open source version of various WebSocket implementations available for both client-side and server-side web development. Sharing exactly the same code for all platforms, with automatic scaling and load-balancing across multiple machines, and just a crumb of code to work - everything out-of-box. There’s nothing more to say.

Don’t hesitate to try it:

// Server
import http from 'http';
import express from 'express';
import SocketIO from 'socket.io-server';

const PORT = 80;

const app = express();
const server = http.Server(app);
const io = new SocketIO(server);

io.on('connection', (socket) => {
    socket.on('ping', () => {
        socket.emit('pong', { alert: 'This is a sample message.' });
    });
});

server.listen(PORT, () => {
    console.log(`Listening on *:${PORT}`);
});

// Client
import SocketIO from 'socket.io-client';

const io = SocketIO();

io.on('pong', (payload) => {
     window.alert(payload.alert);
});

io.emit('ping');

The library I describe above is used by one of the most popular PHP framework applications, Laravel. This is an open source modular web framework that simplifies the creation of web applications and websites using structured development techniques with built in security and authentication capabilities. Laravel is particularly suited for event driven applications.

Laravel Echo was built on the top of Node.js + socket.io and is their flagship real-time communication module. It simplifies WebSocket implementation for fast and scalable messaging between back-end and front-end applications including authenticated private messaging capabilities.

Prototypes & MVPs in Node.js

Fulfilling the client’s requirements in the development of a prototype or MVP usually puts team members under time pressure. When time matters and we don’t need pixel-perfect quality, spending that time on architecture and planning does not lead to success. Node.js, with its npm package manager and thousands of ready-to-use packages, allows a team to focus on useful development.

Usually, at these stages of product development, Node.js can be used with boilerplate projects and default module configurations. Saving time by writing tiny, legible code and spending that saved time on individual needs is an excellent use case of Node.js.

You might be also interested in the article:

Tech

Marble.js – new open source framework for JavaScript

Even more benefits can be achieved with a front-end written in JS. All kinds of APIs (e.g. REST, GraphQL) do not require additional procedures, like transforming JSON-to-object and vice-versa. Accessing all kinds of data looks exactly the same, which gives the ability to reuse code in accordance with coding standards.

Let’s build a very simple REST API, mock-up some predefined data from a .json file and share it via GET endpoint:

import express from 'express';
import loadedData from './mock.json';

const app = express();

app.get('/', (req, res) => res.json({ status: true, message: "Hello world", data: loadedData }));

app.post('/', (req, res) => res.json({ status: false, message: "Exception thrown! Check details argument.", details: "Place your details here" }));

app.listen(port);

Full stack framework for Node.js

With the popularity of Node.js as a web development environment came the creation of framework applications to simplify the web development process. These framework applications are functional development tools that provide common libraries and templates for the automation of repetitive event driven functions and promoting reusability.

The introduction of Node.js to allow the use of JavaScript for server-side applications brought about the concept of full stack JavaScript. Now all web application software written for the web server and the client front-end could be implemented in JavaScript. A full stack framework enables the development of real-time scalable web applications from the front-end user interface through to the back-end server services including data management. The framework is built around libraries of functions which are configured to seamlessly integrate the front-end and back-end applications.

Full stack frameworks available for Node.js range from thin layer additions to Node.js which minimise performance impairment through to more complex and comprehensive full stack framework applications that simplify and automate development and deployment at the cost of real-time performance and reduced scalability. The express framework previously discussed is one of a number of full stack frameworks available for Node.js. It is a lightweight framework that has the advantage of being simple, fast and very well supported.

Command line tools in Node.js

If you need to write a small CLI tool or a big console application, usually you would use the C++ programming language. This language has been used for the development of this kind of project for years. Building an application using C++ requires a lot of time and skill, and the outcome is not always an adequate reward for the necessary effort.

And here comes the Node.js part! The Node.js runtime built on a JavaScript engine offers direct access to most of the useful functions and API layers for I/O or network operations. The lack of needed parts could be filled with repo resources from the npm package manager.

After publishing to the npm package manager, CLI packages are able to download with -g flag. They can be used globally as standard executables built for their operating system. There is no need to add the package to PATH, as it is automatically saved to the Node.js installation directory.

Comparing the C++ programming language and Node.js, the first is better when it comes to performance, but the writing is surely more time-consuming. So, if you prefer a more friendly environment over performance, use Node.js. It is worth adding, that you can cross-use C++ binaries with Node.js tools, which makes things much easier.

Data scraping & automation in Node.js

Unfortunately, not all website owners provide transparent, ready-to-use APIs. Manually copying and processing data periodically wastes a lot of time and is just annoying. The fact is that websites are built with HTML and JavaScript. HTML is simple, structured data, which can be parsed by almost any programming language into its native objects. What about JavaScript? Well, we can manually recreate logical steps based on its code, but is it worthwhile? With Node, we have direct access to read, manipulate and execute JavaScript code in the browser.

The biggest advantage of Node.js, when it comes to data scraping, is that you can test your code directly in any browser.

The libraries and ecosystem of Node.js offer the possibility of accessing a browser as a user, not a dev or machine. An entire DOM built with HTML can be crawled with standard jQuery syntax using the Cheerio library. Adding other modules (e.g. network communication, like request) enables the building of a complex scraping and automation structure. If you want to save time on repeatable steps and just focus on unique functionalities. Voilà!

Let’s load the Google.com site and grab the alternative text for its logo, which is equal to the name of the current international day:

import request from 'request';
import cheerio from 'cheerio';

const url = 'https://google.com/';

request(url, (err, res, body) => {
    if (err) {
        return console.error(err);
    }

    const $ = cheerio.load(body);
    const dayName = $('#hplogo').attribs.alt;

    console.log(`Today we’re celebrating ${dayName}`);
});

Is that everything that Node.js is used for?

Of course not! Due to its flexibility and wide variety of open source packages and full stack framework applications, Node.js has almost the unlimited ability to make your development dreams come true. With its high level of scalability, it can be used with applications of various sizes and needs. From small utility tools to a big, complex web server, Node.js is always lightweight and developer friendly. Long story short, Node.js is that type of technology which just works without unnecessary effort. Yes, it may have some disadvantages (e.g. big data computing) but it is still worth giving it a try.