Autoscaling: horizontal vs vertical

What Does Auto Scaling Mean and Why Do You Need It?

What happens when your website or service traffic suddenly multiplies? If your infrastructure is managed manually, you will either face severe slowdowns or, in the worst case, the service goes down. Scalability refers to the system's ability to respond to increased load, and Auto Scaling means this process happens without human intervention and based on predefined rules.

There are two main approaches to scalability: Horizontal and Vertical. Choosing between these two is one of the most important architectural decisions that directly impacts the cost, complexity, and stability of your service. In this article, we will examine both approaches in technical detail, outline the architectural prerequisites for each, and finally, show common pitfalls that teams usually fall into, with real-world examples.

Vertical Scaling: Simple but Limited

In vertical scaling, you increase the power of a single server: more RAM, a stronger CPU, faster storage (NVMe), or a higher-speed network. This method is also called Scale Up. For example, if your server has 4 CPU cores and 8 GB of RAM, you upgrade it to 16 cores and 64 GB of RAM.

Advantages of Vertical Scaling

• Simplicity: No changes to the application code or architecture are required. The same server just becomes more powerful.
• Lower operational costs: You don't need to manage multiple servers, a Load Balancer, or data synchronization between nodes.
• Full compatibility: It may be the only option for Legacy applications that were originally designed for single-instance execution.

Disadvantages and Limitations of Vertical Scaling

• Hardware ceiling: Every server has physical limits. You cannot upgrade a server indefinitely. Even the largest available servers have a specific limit.
• Single Point of Failure: If that one server experiences a hardware issue, the entire service goes down.
• Non-linear cost: The price of large servers increases exponentially. A server with 64 cores usually costs several times more than a 16-core server, not just 4 times more.
• Downtime for upgrades: In most cases, upgrading resources requires a server restart, which means a few minutes of service interruption.

When is Vertical Scaling the Right Choice?

If your traffic is relatively predictable, doesn't have extreme fluctuations, and your application is written as a single instance (like many traditional PHP or Node.js applications), vertical scaling is the simplest and most cost-effective option. Also, for relational databases that require strong transactions, vertical scaling is often the first choice.

Horizontal Scaling: Distributed Power

In horizontal scaling, instead of making one server more powerful, you increase the number of servers. This method is called Scale Out. If one server can handle 1000 requests per second, with 5 servers, you can manage 5000 requests. This approach is the foundation of modern Cloud-Native architectures.

Advantages of Horizontal Scaling

• Almost unlimited scalability: As long as the network infrastructure allows, you can add more servers.
• High fault tolerance: If one server fails, the others continue operating, and traffic is distributed among them.
• Linear cost: The cost per unit (server) is fixed. 10 servers cost exactly 10 times more than one server, not more.
• Flexibility in auto scaling: Adding or removing a server is far simpler than upgrading hardware and is usually done without downtime.

Challenges of Horizontal Scaling

• Architectural complexity: Your application must be Stateless, meaning no data is stored in the server's local memory. Sessions must be stored in Redis or a central database.
• Need for a Load Balancer: You need a load distribution layer to distribute traffic among servers.
• Data management: If you use a database, you must choose between Replication and Sharding, each with its own complexities.
• Harder debugging: When an error occurs, you need to check logs from multiple servers and trace Correlation IDs.

Architectural Prerequisites for Horizontal Scaling

To be able to use horizontal scaling, your architecture must meet the following conditions:

1. Statelessness: No data should be stored in the local filesystem or server memory. Uploaded files should be placed on Object Storage like S3 or MinIO.
2. Centralized Session Management: Sessions should be stored in Redis or Memcached, not in the server's own memory.
3. Scalable Database: The database must be configured as Master-Slave or a Cluster. For heavy write loads, Sharding is essential.
4. Health Check: The Load Balancer must be able to check the health of each server and remove unhealthy servers from the rotation.

Practical Comparison: Which Approach is Right for You?

To decide, you need to consider three main factors: traffic pattern, application nature, and budget.

Traffic Pattern

If your traffic has extreme fluctuations (like an online store on special nights, or a news service during major news releases), horizontal auto scaling is the best option. You can define a rule that if CPU goes above 70%, a new server is added. With vertical scaling, this is practically impossible because hardware upgrades are time-consuming and you cannot quickly scale down.

Application Nature

Real-time applications like WebSocket or online games make horizontal scaling harder due to the need to maintain connection state. In these cases, Sticky Sessions are usually used, which have their own limitations. Batch Processing and Queue-based applications usually scale horizontally with ease.

Costs

For stable, low-fluctuation workloads, vertical scaling is often cheaper because you don't have the cost of a Load Balancer and the complexity of managing multiple servers. However, for variable workloads, horizontal scaling is more economical because you can reduce the number of servers during low-demand hours and pay less.

Implementing Auto Scaling with a Practical Example

Suppose you have a web service running on Linux servers and you want to implement horizontal auto scaling. The main steps are as follows:

1. Setting up a Load Balancer

First, you need a Load Balancer. You can do this easily with Nginx:

upstream backend {
    least_conn;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.13:8080 max_fails=3 fail_timeout=30s;
}

server {
    listen 80;
    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

In this configuration, Nginx distributes traffic among three servers, and if a server fails three consecutive times, it is removed from the rotation for 30 seconds.

2. Defining an Auto Scaling Rule

For auto scaling, you can use tools like Kubernetes Horizontal Pod Autoscaler or Auto Scaling services from cloud providers. In Kubernetes, defining a simple rule looks like this:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

This rule states that if the average CPU utilization of the pods goes above 70%, the number of pods should be increased (up to a maximum of 10), and if it drops, it should be decreased (to a minimum of 2).

3. Database Management

If you are using MySQL, you need to set up Replication for horizontal scaling. A simple example of Master-Slave configuration:

-- On the Master server
[mysqld]
server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
binlog_do_db = myapp

-- On the Slave server
[mysqld]
server-id = 2
relay-log = /var/log/mysql/mysql-relay-bin.log

Then, on the Slave, run the following command:

CHANGE MASTER TO
  MASTER_HOST='10.0.0.20',
  MASTER_USER='replica',
  MASTER_PASSWORD='secret',
  MASTER_LOG_FILE='mysql-bin.000001',
  MASTER_LOG_POS= 107;
START SLAVE;

Important note: Only direct reads (SELECT) to the Slaves and keep writes (INSERT/UPDATE) on the Master.

Common Pitfalls in Auto Scaling

Many teams encounter unexpected issues after implementing auto scaling. Here, we examine the most common pitfalls:

Pitfall 1: Ignoring Session and State

Suppose your application stores sessions in local memory. When the Load Balancer routes a user's request to another server, the user gets logged out. Solution: Move sessions to Redis. A simple example with Node.js and Express:

const session = require('express-session');
const RedisStore = require('connect-redis')(session);

app.use(session({
    store: new RedisStore({ host: 'redis.internal', port: 6379 }),
    secret: 'your-secret-key',
    resave: false,
    saveUninitialized: false
}));

Pitfall 2: Forgetting to Scale the Database

Many teams only scale application servers but keep the database on a single server. The result: the application gets faster, but the database becomes the bottleneck. If your database reaches 100% CPU, adding more application servers won't help at all. You must scale the database first.

Pitfall 3: Thrashing

If you set the scaling rule too sensitively, the system may constantly add and remove servers. This causes extra costs and instability. To prevent this, use a Cooldown Period. In Kubernetes, you can increase the --horizontal-pod-autoscaler-downscale-stabilization parameter to prevent rapid pod removal.

Pitfall 4: Ignoring Cold Start

When a new server is added, it takes a few minutes for the application to start up and for its cache to warm up. If your rule reacts too late, the service will encounter errors during those first few minutes. Solution: Use Proactive Scaling based on time or calendar events. For example, if you know traffic increases every night at 8 PM, add servers in advance.

Summary and Final Recommendation

Choosing between horizontal and vertical scaling is not an all-or-nothing decision. Many successful systems use a combination of both: vertical scaling for the main database and horizontal scaling for the application layer. The key point is not to treat scalability as a late-stage feature; if you design a Stateless architecture and a Replication-ready database from the start, you will have more options in the future.

If you are just starting out and your traffic is low, start with vertical scaling. But as soon as you approach the hardware ceiling or your traffic fluctuations increase, plan the migration to horizontal scaling. Along the way, cloud infrastructure management tools can be very helpful; for example, the cloud services offered by ServerNet allow you to define Auto Scaling rules natively, so your infrastructure can keep up with traffic without manual management.

Finally, remember that scalability is not just about hardware; it's about proper design, continuous monitoring, and being prepared for failure. Build your system so that the failure of one component doesn't bring down the entire system.