How to Setup Nginx as loadbalancer for Apache or Tomcat for HTTP/HTTPS

You can use Nginx as a loadbalancer in front of your web application.

For example, if your enterprise application is running on Apache (or Tomcat), you can setup an 2nd instance of your enterprise application on Apache (or Tomcat) on a different server.

And then, you can put Nginx at the front-end, which will load balance between the two Apache (or Tomcat, or JBoss) servers.

If you are new to Nginx, it is important to understand the difference between Nginx vs Apache, and Nginx Architecture.

Nginx supports the following three types of load balancing:

1. round-robin – This is the default type for Nginx, which uses the typical round-robin algorithm to decide where to send the incoming request
2. least-connected – As the name suggests, the incoming request will be sent to the server that has the less number of connection.
3. ip-hash – This is helpful when you want to have persistence or stick connection of the incoming request. In this type, the client’s ip-address is used to decide which server the request should be sent to.

1. Define upstream and proxy_pass in Nginx Config File

For load balancing, you need to add two things to the nginx configuration file: 1) upstream 2) proxy_pass

First, upstream: Specify a unique name (may be name of your application) and list all the servers that will be load-balanced by this Nginx.

In the following example, “crmdev” is the name of the upstream, which is the name of the application that is running on both the individual Apache webserver (101.1 and 102.2 as shown below). Instead of “crmdev”, you can specify anything that you like here.

Note: The upstream should be defined inside your “http” context of Nginx.

upstream crmdev {
    server 192.168.101.1;
    server 192.168.101.2;
}

Note: If the individual servers are running on a different port (other than port 80), then specify the port number as shown below in the upstream

upstream crmdev {
    server 192.168.101.1:8080;
    server 192.168.101.2:8080;
}

Second, proxy_pass: Specify the unique upstream name that was defined in the previous step as proxy_pass inside your “location” section, which will be under “server” section as shown below.

server {
  listen 80;
  
  location / {
    proxy_pass http://crmdev;
  }
}

Note: In this example, nginx itself is listening on port 80 as shown above by the listen parameter.

Please note that you can also use proxy_pass to Setup Nginx as a Reverse Proxy to Apache/PHP.

2. Define upstream and proxy_pass in Nginx Default Config File

Note: Typically, the above should be under http or https as shown below.

http {
  upstream crmdev {
   server 192.168.101.1:8080;
   server 192.168.101.2:8080;
  }

  server {
      listen 80;

      location / {
          proxy_pass http://crmdev;
      }
  }
}

But, if you are using the default.conf that comes with the default nginx.conf file, you don’t need to give the “http”, as it is already defined in the http context.

Note: For HTTPS, replace the “http” context (in the 1st line above) with https. Also, in the proxy_pass line, use https://{your-upstream-name}

In this case, if you use the “http” as shown above, you might get the following http directive is not allowed error message:

Starting nginx: nginx: [emerg] "http" directive is not allowed here in /etc/nginx/conf.d/default.conf:1

This is the copy of my default.conf file, where I’ve added upstream at the top (without the http), and then commented out the default location, and added the new location using the proxy_pass.

# vi /etc/nginx/conf.d/default.conf
upstream crmdev {
    server 127.0.0.1:4080;
    server 127.0.0.1:7080;
}
server {
  listen       3080;
  server_name  localhost;
  
  #location / {
  #    root   /usr/share/nginx/html;
  #    index  index.html index.htm;
  #}
  
  location / {
    proxy_pass http://crmdev;
  }
..
..
}

3. Setup least-connected Algorithm for Nginx Load Balancer

In this algorithm, the incoming request is sent to the server that has least number of existing active connection.

For this, add the keyword “least_conn” at the top of the upstream as shown below.

upstream crmdev {
    least_conn;
    server 192.168.101.1:8080;
    server 192.168.101.2:8080;
}

If you have several servers listed in the least_conn, and if multiple servers are having the similar low number of existing active connections, then among those servers, one will be picked based on weighted round-robin.

4. Setup Persistence or Sticky Algorithm for Nginx Load Balancer

The disadvantage of the round-robin and least-connected method is that, the subsequent connection from the client will not go to the same server in the pool. This may be Ok for a non session dependent application.

But if your application is dependent on session, then once an initial connection is established with a particular server, then you want all future connection from that particular client to go to the same server. For this, use the ip_hash algorithm as shown below.

upstream crmdev {
    ip_hash;
    server 192.168.101.1:8080;
    server 192.168.101.2:8080;
} 

For the hash, for IPv4 address, first three octets are used. If it is a IPv6 address, the entire address is used.

5. Weight Options for the Individual Servers

You can also specify a weight for a particular server in your pool. By default, all the servers has the equal priority (weight). i.e The default value of weight is 1.

But, you can change this behavior by assigning a weight to a server as shown below.

upstream crmdev {
    server 192.168.101.1:8080;
    server 192.168.101.2:8080;
    server 192.168.101.3:8080 weight 2;
    server 192.168.101.4:8080;
    server 192.168.101.5:8080;
}

In this example, we have total of 5 servers. But the weight on 3rd server is 2. This means that for every 6 new request, 2 request will go to 3rd server, and rest of the server will get 1 request.

So, this is helpful to distribute more load on a specific server which has more horsepower.

Eventhough in the above example, weight is used with the default round-robin algorithm, you can use weight for least_conn and ip_hash also.

6. Timeout Options for the Individual Servers – max_fails and fail_timeout

You can also specify max_fails and fail_timeout to a particular server as shown below.

upstream crmdev {
    server 192.168.101.1:8080 max_fails=3 fail_timeout=30s;
    server 192.168.101.2:8080;
    server 192.168.101.3:8080 weight 2;
    server 192.168.101.4:8080;
    server 192.168.101.5:8080;
}

In the above:

• The default fail_timeout is 10 seconds. In this above example, this is set to 30 seconds. This means that within 30 seconds if there were x number of failed attempts (as defined by max_fails), then the server will be unavailable. Also, the server will remain unavailable for 30 seconds.
• The default max_fails is 1 attempt. In the above example, this is set to 3 attempts. This means that after 3 unsuccessful attempts to connect to this particular server, Nginx will consider this server unavailable for the duration of fail_timeout which is 30 seconds.

7. Assign a Backup Server in Nginx LoadBalancer Pool

In the following example, the 5th server is marked as backup using “backup” keyword at the end of server parameter.

upstream crmdev {
    server 192.168.101.1:8080 max_fails=3 fail_timeout=30s;
    server 192.168.101.2:8080;
    server 192.168.101.3:8080 weight 2;
    server 192.168.101.4:8080;
    server 192.168.101.5:8080 backup;
}

The above will make the 5th server (192.168.101.5) as a backup server. Incoming request will not be passed to this server unless all the other 4 servers are down.