Git, Linux & DevOps
Performance and Scalability
Performance is how fast a system responds under a given load; scalability is whether it keeps performing acceptably as that load - more users, more data, more requests - grows. They're related but distinct: a system can be fast today and fail to scale, slowing to a crawl at ten times the traffic, or scale well while never being particularly fast for any individual request. Improving one doesn't automatically improve the other, which is why both need to be measured and reasoned about separately.
Why it matters
- Slow systems lose users, regardless of how correct the underlying code is
- A feature that works perfectly but takes several seconds to respond is, in practice, a broken feature for most users.
- Scaling problems tend to appear suddenly, not gradually
- A system can perform fine for months and then fail sharply once a resource - a database's connection limit, a queue's throughput - is exceeded, because many bottlenecks behave fine right up to a hard limit.
- The fix for a performance problem and the fix for a scalability problem are often different
- Making one request faster is not the same engineering problem as handling ten times as many concurrent requests, and the wrong fix wastes real effort.
- Measuring correctly is most of the work
- Optimizing without profiling first routinely improves the wrong part of the system while the actual bottleneck goes untouched.
Latency vs throughput
Latency is how long one request takes from start to finish. Throughput is how many requests the system handles per unit of time. They're related but not the same: a system can have low latency but low throughput if it can only handle a few requests at once, or the reverse, where batching makes each individual item slightly slower to process but lets the system handle far more overall.
Vertical vs horizontal scaling
Vertical scaling means moving to a bigger machine - more CPU, more memory - which is simple and requires no architectural change, but has a hard ceiling (the biggest machine available) and no redundancy, since it's still one machine. Horizontal scaling means adding more machines and spreading load across them, which removes both limits but requires the application to actually support sharing load, usually meaning it's stateless or externalizes any state it needs to a shared store.
Where the bottleneck usually is
Guessing at what's slow is frequently wrong; profiling - measuring where time is actually spent - finds the real bottleneck far more reliably than intuition does. In practice, a large share of real-world slowness traces back to a small set of causes: unindexed database queries, fetching related data one row at a time instead of in a batch, and blocking I/O that holds up other work while it waits.
-- N+1: one query per order, run 500 times for 500 orders
SELECT * FROM customers WHERE id = 42;
SELECT * FROM customers WHERE id = 91;
-- ...498 more...
-- batched: one query for all needed customers
SELECT * FROM customers WHERE id IN (42, 91, 137);Mistakes people make here
- Optimizing before measuring
- intuition about what's slow is frequently wrong; profiling first, then optimizing the part that's actually the bottleneck, is far more effective than guessing.
- The N+1 query problem
- fetching a list, then querying again for each item's related data in a loop, turns one page load into hundreds of round-trips to the database; batching those into one query is usually a large, easy win.
- Assuming horizontal scaling is free once you add more servers
- it only works if the application can actually share load across servers, which usually means being stateless or externalizing state somewhere shared; a server that keeps state in memory doesn't scale horizontally just by adding copies of it.
- Caching without a plan for invalidation
- a cache that serves stale data because nothing tells it when to update is often worse than no cache at all, since the bug is silent instead of the system just being slow.
- Treating a load test as optional
- a system that has never been tested under realistic concurrent load has an unknown scaling ceiling, and the first time it's found out is often during real peak traffic.
Strengths and trade-offs
Where it is strong
- Caching, indexing and batching are well-understood techniques that often produce large wins for relatively small, targeted effort.
- Horizontal scaling, once an application is built to support it, can handle load growth by adding capacity rather than needing a fundamental rewrite.
- Profiling tools make it possible to find the actual bottleneck rather than relying on guesswork.
The trade-offs
- Optimizations often trade simplicity for speed - a cache, a denormalized table, or an async pipeline all add moving parts and failure modes a simpler, slower version didn't have.
- Scaling horizontally requires architectural discipline decided early; retrofitting it into a system built assuming a single server is a substantial rework.
- Chasing performance past the point it matters to users wastes engineering time that could go elsewhere - not every millisecond is worth pursuing.
Who needs this
Backend engineers and anyone responsible for a system under real user load need this directly. Frontend engineers need a lighter version of it - perceived performance, network waterfall, bundle size - which shares the same underlying discipline of measuring before optimizing.
Questions about performance and scalability
- What's the difference between latency and throughput?
- Latency is how long one request takes from start to finish. Throughput is how many requests the system handles per unit of time. A system can have low latency but low throughput if it can only handle a few at once, or the reverse, where batching makes each individual item slightly slower but the system processes far more overall.
- Should I scale vertically or horizontally?
- Vertical scaling is simpler and often the right first move, since it requires no architectural change, but it has a hard ceiling and no redundancy. Horizontal scaling removes both limits but requires the application to be built to share load across machines, which is real, upfront design work.
- What is the N+1 query problem?
- It's a common pattern where code fetches a list of items with one query, then loops over the list and runs a separate query for each item's related data, turning what should be one or two queries into hundreds. It's usually fixed by fetching the related data in a single batched query instead of one per item.
- How do I know if my system will scale before it's under real load?
- Load testing - deliberately simulating higher concurrent traffic than the system normally sees, and watching where it starts to slow down or fail - is the standard way to find a scaling ceiling before real users do. It won't catch everything, but it catches far more than reasoning about it without ever testing.