NATS as an Alternative to Kafka and RabbitMQ

Integrating NATS with Express.js and Containerizing with Docker Compose and Kubernetes

Introduction

NATS is a lightweight, high-performance, easy-to-use messaging system designed for simplicity and speed. It has gained popularity as a simple alternative to more complex messaging systems like Apache Kafka and RabbitMQ. In this article, we will explore how NATS can replace Kafka and RabbitMQ, demonstrate how to integrate NATS with an Express.js application and containerize the solution using Docker Compose and Kubernetes.

Comparing NATS to Kafka and RabbitMQ

While Apache Kafka and RabbitMQ are powerful and feature-rich messaging systems, they can be overkill for certain use cases. NATS aims to provide a simpler, more lightweight solution that’s easy to set up and use while maintaining high performance.

NATS vs. Kafka

• Simplicity: NATS is designed for simplicity, making it easier to set up and use compared to Kafka, which requires more configuration and management overhead.
• Performance: NATS is built for high performance and low latency, whereas Kafka is designed for high-throughput and durability.
• Durability: Kafka provides strong durability guarantees by persisting messages on disk, while NATS is more focused on simplicity and performance, making it more suitable for use cases that don’t require message persistence.

NATS vs. RabbitMQ

• Simplicity: NATS is easier to set up and use than RabbitMQ, which has a more complex configuration and management process.
• Performance: NATS is designed for high performance and low latency, while RabbitMQ offers a broader set of features such as advanced message routing and message persistence.
• Routing: RabbitMQ provides more advanced routing capabilities, such as topic-based routing, while NATS has a simpler publish-subscribe model.

Prerequisites

• Basic understanding of Node.js and Express.js
• Familiarity with Docker and Docker Compose
• Basic knowledge of Kubernetes

Step 1: Creating the Publisher and Subscriber Servers

Let’s create a new directory for the project:

mkdir express-nats-example && cd express-nats-example

Now let’s create a new directory for the publisher server and navigate to it:

mkdir express-nats-publisher && cd express-nats-publisher

Now we’ll initialize nodejs project and install required npm packages:

npm init -y
npm install express nats

We will create a publisher server in a file named publisher.js:

const express = require('express');
const NATS = require('nats');
const app = express();
const port = process.env.PORT || 3001;

// Connect to NATS server
const nc = NATS.connect('localhost:4222'); // Replace with your NATS server address if not running locally

app.get('/publish/:message', (req, res) => {
  const message = req.params.message;
  const subject = 'mySubject';

  nc.publish(subject, message);
  res.send(`Message sent: ${message}`);
});

app.listen(port, () => {
  console.log(`Publisher server listening on port ${port}`);
});

Now we need to create our subscriber service. We’ll create a new directory for the subscriber server in our project’s folder and navigate to it:

mkdir express-nats-subscriber && cd express-nats-subscriber

Initialize the Node.js project and install dependencies:

npm init -y
npm install express nats

Everything is ready to create our subscriber service a file named subscriber.js:

const express = require('express');
const NATS = require('nats');
const app = express();
const port = process.env.PORT || 3002;

// Connect to NATS server
const nc = NATS.connect('localhost:4222'); // Replace with your NATS server address if not running locally

const subject = 'mySubject';

nc.subscribe(subject, (message) => {
  console.log(`Received message: ${message}`);
});

app.get('/', (req, res) => {
  res.send('Subscriber server is running.');
});

app.listen(port, () => {
  console.log(`Subscriber server listening on port ${port}`);
});

Step 2: Containerizing publisher and subscriber Applications with Docker

We need to create 2 Dockerfile files in each of our applications. The contents of the file will be the same:

FROM node:lts

WORKDIR /usr/src/app

COPY package*.json ./

RUN npm install

COPY . .

EXPOSE 3000

CMD ["node", "app.js"]

We’ll also need to create .dockerignore files to exclude unnecessary files from the docker image:

node_modules
npm-debug.log

Step 3: Setting Up Docker Compose

Let’s create the docker-compose.yml file in the root of our express-nats-example folder:

version: '3.8'

services:
  publisher:
    build: ./express-nats-publisher
    ports:
      - "3001:3001"
  subscriber:
    build: ./express-nats-subscriber
    ports:
      - "3002:3002"
  nats:
    image: "nats:2.6"
    ports:
      - "4222:4222"

We can test our docker-compose file by running:

docker-compose up

This should start our publisher, subscriber and nats servers

Now we have two separate Express.js servers that communicate with each other through NATS. We can publish messages by sending a GET request to http://localhost:3001/publish/<message>, and the subscriber server will receive and log the messages.

subscriber_1  | Subscriber server listening on port 3002
subscriber_1  | Received message: Hello
subscriber_1  | Received message: How_are_you
subscriber_1  | Received message: Goodbye

Step 4: Deploying the Application with Kubernetes

Let’s take it one step further and deploy our apps to Kubernetes.

To do so we have to build docker images for our publisher and subscriber applications:

docker build -t express-nats-publisher ./express-nats-publisher
docker build -t express-nats-subscriber ./express-nats-subscriber

Let’s create a kustomization.yml file in the root of our project that will deploy our Kubernetes resources:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - deployment.yml
  - publisher-service.yml
  - subscriber-service.yml

Now we need to create a deployment.yml file to define the Kubernetes deployment for our services:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: express-nats-example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: express-nats-example
  template:
    metadata:
      labels:
        app: express-nats-example
    spec:
      containers:
        - name: publisher
          image: express-nats-publisher
          ports:
            - containerPort: 3001
        - name: subscriber
          image: express-nats-subscriber
          ports:
            - containerPort: 3002
        - name: nats
          image: nats:2.6
          ports:
            - containerPort: 4222

We also need to create a publisher-service.yml as well as subscriber-service.yml file to be able to access both our publisher and subscriber:

apiVersion: v1
kind: Service
metadata:
  name: express-nats-publisher
spec:
  selector:
    app: express-nats-example
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3001
  type: LoadBalancer

apiVersion: v1
kind: Service
metadata:
  name: express-nats-subscriber
spec:
  selector:
    app: express-nats-example
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3002
  type: LoadBalancer

After creating all necessary files we’re ready to deploy to Kubernetes

kubectl apply -k .

Now we can access the application using the external IP addresses assigned by the LoadBalancers

kubectl get services

Use Cases for NATS over Kafka and RabbitMQ

NATS is a great choice for specific use cases where the simplicity of its setup, lower resource usage, and ease of use are important factors. Some of these scenarios might include:

1. IoT: With its low latency and lightweight design, NATS is an excellent choice for IoT applications, where devices need to communicate efficiently and quickly.
2. Real-time updates: NATS can be used for real-time updates and notifications, such as broadcasting messages to clients in a web or mobile application.
3. Microservices: NATS is a suitable choice for microservices communication where simplicity, speed, and ease of integration are crucial.
4. Simple publish-subscribe: If you need a basic publish-subscribe messaging system without advanced routing or message persistence, NATS is a good choice.

However, if you require advanced features like message durability, complex routing, or log-based stream processing, Apache Kafka and RabbitMQ can provide more comprehensive solutions tailored to those needs. It’s essential to analyze your specific use case and requirements to choose the right messaging system for your application.

Conclusion

In this article, we have explored how NATS can serve as an alternative to more complex messaging systems like Apache Kafka and RabbitMQ. We demonstrated how to integrate NATS with an Express.js application and containerize the solution using Docker Compose and Kubernetes. By following these steps, you can create a scalable and efficient messaging system for your Express.js applications using NATS, which can be especially beneficial in scenarios where simplicity and ease of use are more important than advanced features like message persistence and complex routing.