What does Scalability mean?

Scalability describes a system’s ability to keep working reliably as load grows, given additional resources. It shows up in whether doubling compute power also roughly doubles throughput. It differs from raw performance in that what matters isn’t a one-off peak value, but how the system behaves as load increases.

With vertical scaling, a single server gets more memory, more processor cores, or a stronger graphics card, until the chassis runs out of room. Horizontal scaling instead spreads load across multiple servers, which requires a load balancer and services that don’t hold session state in memory. Containers and Kubernetes make it possible to add capacity during operation and remove it again afterward. For AI applications, the bottleneck usually sits with GPU memory, less often with compute time.

The question of scalability arises as soon as a service opens up beyond a single department and load fluctuates over the course of the day. Examples include search services, workplace assistance functions, and reporting tied to fixed dates. With a steady number of users and requests, a fixed setup is enough.

The advantage over sizing for peak load lies in cost. Capacity is added during spikes and then switched off again, so no idle hardware gets paid for outside busy periods. Additional areas can be connected without changing the architecture.

Scalability is decided by architecture, not by procurement. If a service stores its users’ session data in a single server’s memory, every request has to return to that server, and additional servers achieve nothing. Running in an organization’s own data center adds its physical limit on top, which should be known before any procurement decision.