What a load balancer is and how it works

What is a load balancer and why do you need it?

Imagine your website suddenly faces a surge in traffic. If you have only one server, that server will quickly become saturated, and users will encounter 503 Service Unavailable errors or severe slowness. The classic solution is to place multiple servers behind a load balancer. A load balancer is a device (software or hardware) that distributes incoming traffic among several servers so that no single server is overloaded beyond its capacity.

The main purpose of a load balancer goes beyond just "spreading traffic." A good load balancer must do three things simultaneously: load distribution based on an appropriate algorithm, continuous health checks of servers, and session management (Session Persistence) for users with consecutive requests. Without any of these three, your infrastructure will be unstable in practice.

In this article, in simple but precise language, you will learn about the types of load balancers, load distribution algorithms, how to configure health checks, and session management techniques. If you are designing an infrastructure that needs to be scalable, this guide will help you make the right decision.

Types of Load Balancers: Hardware, Software, and DNS

Load balancers can be divided into three main categories. Choosing each one depends on your budget, scale, and technical requirements.

Hardware Load Balancer

These are physical devices such as F5 BIG-IP or Citrix ADC, designed specifically for load distribution. Their main advantage is very high performance and advanced features like SSL termination and layer 7 firewall. However, their purchase and maintenance costs are high, and their scalability is limited to the hardware. For small and medium businesses, they are usually considered an uneconomical option.

Software Load Balancer

Tools like NGINX, HAProxy, and Traefik that are installed on a regular server. This option is the most popular choice for startups and medium-sized companies. HAProxy is especially powerful for layer 4 (TCP/UDP) and layer 7 (HTTP/HTTPS) load balancing. NGINX, in addition to being a load balancer, can also act as a web server and reverse proxy. Their installation and configuration are relatively simple, and they have good documentation.

DNS Load Balancer

In this method, DNS is used for load distribution. A domain responds with multiple IPs, and the client randomly selects one. Services like AWS Route 53 or Cloudflare use this method. Its advantage is simplicity and no need for additional infrastructure, but it has serious drawbacks: if one of the servers goes down, DNS will still return its IP until the TTL expires. Also, it does not provide precise control over load distribution and is not suitable for session-sensitive applications.

In modern cloud infrastructures, a combination of software load balancers (for internal traffic) and DNS (for regional failover) is typically used. If you are just starting to set up a service, I recommend starting with HAProxy or NGINX; these tools can also be easily installed on ServerNet cloud servers.

Load Distribution Algorithms: Which One Is Right for You?

The beating heart of any load balancer is the algorithm that decides which server each request should be sent to. Choosing the wrong algorithm can cause some servers to be overworked while others remain idle. In the following, we will examine the most important algorithms.

Round Robin and Weighted Round Robin

The simplest algorithm is Round Robin, which distributes requests cyclically among servers: the first request goes to server A, the second to B, the third to C, and then back to A. This method is excellent for servers with equal capacity. However, if servers have different capacities, use Weighted Round Robin. You assign a weight to each server; for example, if server A has weight 3 and server B has weight 1, then out of every 4 requests, 3 go to A and 1 goes to B.

Example configuration in HAProxy:

backend web_servers
    balance roundrobin
    server web1 192.168.1.10:80 weight 3 check
    server web2 192.168.1.11:80 weight 1 check

Least Connections

This algorithm sends the request to the server that currently has the fewest active connections. It is very suitable for applications with long-running requests (such as file uploads or WebSocket). In HAProxy, it is enabled with the balance leastconn directive. This method is smarter than Round Robin, but it requires continuous monitoring of connection counts.

IP Hash and Source IP Affinity

In this method, the client's IP address is converted into a hash number, and based on that, a specific server is selected. The main advantage is that a user is always directed to the same server they first connected to. This is useful for applications that store sessions on the server (such as login with in-memory sessions). However, if the number of clients is small, load distribution may become uneven.

More Advanced Algorithms

Some load balancers have more complex algorithms, such as Least Response Time or Consistent Hashing, which is used for distributed caches. These algorithms are usually available in commercial tools or cloud services. To get started, Round Robin and Least Connections are sufficient.

Health Check: Ensuring Service Stability

A load balancer without health checks is just a blind distributor. If a server goes down, the load balancer will still send requests to it, and users will see errors. A health check is a mechanism by which the load balancer periodically verifies the health of servers and removes unhealthy servers from the rotation.

Types of Health Checks

• TCP Check: The load balancer establishes a TCP connection to a specific port (e.g., 80 or 443). If the connection is successful, the server is healthy. This is the simplest method, but it only checks that the port is open, not whether the application is actually working.
• HTTP Check: The load balancer sends an HTTP request to a specific path (e.g., /health) and checks that the response returns with a 200 status code. This method is more accurate and can also check the status of database connections or other services.
• Custom Script Check: For specific needs, you can write a script that thoroughly checks the health of the application. For example, it can check that the message queue is not full.

Example HTTP health check configuration in NGINX:

upstream backend {
    server 192.168.1.10:80 max_fails=3 fail_timeout=30s;
    server 192.168.1.11:80 max_fails=3 fail_timeout=30s;
}

server {
    location / {
        proxy_pass http://backend;
        proxy_next_upstream error timeout http_502;
    }
}

In this example, if a server fails three times within 30 seconds, it is temporarily removed from the rotation. The proxy_next_upstream parameter causes the request to be sent to the next server if the first server returns a 502 error.

Session Management (Session Persistence)

Many web applications store user session information on the server. If a user connected to server A sends their next request to server B, their session will be lost, and they will have to log in again. To solve this problem, session management is used, also known as Sticky Session or Session Affinity.

Methods for Implementing Sticky Sessions

• Cookie-based: The load balancer sets a cookie named SERVERID in the user's browser. In subsequent requests, the user sends this cookie, and the load balancer directs the user to the same server based on it. This method is enabled in HAProxy with the cookie SERVERID insert indirect option.
• Source IP Affinity: This is the IP Hash algorithm we explained earlier. It is simple, but if users are behind a NAT or proxy, they all share one IP and will be directed to a single server.
• Session Replication: Instead of sticking the user to one server, you replicate the session across all servers. This method is complex but offers better scalability. Tools like Redis or Hazelcast are used for this purpose.

Example of Sticky Session in HAProxy:

backend web_servers
    balance roundrobin
    cookie SERVERID insert indirect nocache
    server web1 192.168.1.10:80 cookie web1 check
    server web2 192.168.1.11:80 cookie web2 check

Summary and Practical Recommendations

Choosing the right load balancer and algorithm depends on your needs. To get started, I recommend this path:

1. If you have a small infrastructure (fewer than 5 servers), start with HAProxy and the Round Robin algorithm.
2. Definitely enable health checks from day one, even if you have only one server.
3. If your application has sessions, first try to make it stateless (using Redis). If that is not possible, use Sticky Sessions.
4. For very high traffic, set up the load balancer in an Active-Passive configuration so that the load balancer itself is not a single point of failure.

Finally, remember that a load balancer is just one component of a scalable infrastructure. Monitoring, caching, and database optimization are equally important. If you are just starting out, open-source tools like HAProxy and NGINX are a smart choice and can be easily deployed on ServerNet cloud services. With proper configuration, your infrastructure can remain stable without worrying about traffic fluctuations.