Scalability

HTTP caching (Cache-Control, ETag, fresh/stale, immutable static resources); reverse proxies; application-layer: side vs inline cache, LRU eviction, TTL, local vs external cache, thundering herd, cascading failure; 80% hit rate as scalability threshold

CDN overlay network; BGP limitations; global DNS LB; IXP placement; edge+intermediary caching layers; DDoS shielding

data plane (on critical path, availability), control plane (off-path, consistency); static stability; scale imbalance solutions (file store buffer, push deltas, hybrid); control theory feedback loop

*Foundations of Scalable Systems* — Ian Gorton

DNS LB, L4 (transport), L7 (application), service discovery, health checks, power of two choices, sidecar as client-side LB; stateless services as prerequisite for scale-out

When to use ML (nine conditions); research vs production differences (silent failure, latency vs throughput, messy data, fairness, interpretability); four system requirements (reliability, scalability, maintainability, adaptability); business vs ML objective alignment; problem framing (task types, decoupling objectives); mind vs data debate

Key range, hash, consistent hashing, secondary indexes (local/global), rebalancing strategies, request routing; cross-partition complexity costs

Symptom → diagnosis → action playbook; three analytical lenses (Little's Law, utilisation curve, percentile arithmetic); capacity planning workflow; antipatterns (sizing for average, running at high utilisation, unbounded queues); when performance tuning doesn't apply

Little's Law (L=λW), utilisation curves (W=S/(1−ρ)) and hyperbolic response-time growth, tail latency amplification in fan-out designs, percentile arithmetic (averages lie; can't average percentiles; t-digest/HdrHistogram), queues in series, thread pools as queueing systems; practical heuristics for capacity and tuning

Load shedding (503, priority/age ordering), load leveling (async channel + auto-scaling), rate limiting (sliding window buckets, distributed atomic increment, fail-open), constant work pattern (periodic full-state dump, antifragile, self-healing)

State machine replication (Raft), chain replication (head/tail topology, failure modes, data/control plane split), leader-follower, Dynamo-style; replication lag anomalies; multi-leader conflict resolution

definition; replication vs optimization strategies; scale up vs scale out; stateless services requirement; Amdahl's Law; hyperscale; quality attribute trade-offs (performance, availability, security, manageability); architecture evolution pattern

serverless model (pay-per-invocation, managed autoscaling); cold start by runtime; GAE autoscaling parameters; AWS Lambda (freeze/thaw, provisioned/reserved concurrency, burst limits); parameter study methodology; vendor lock-in

*Understanding Distributed Systems* — Roberto Vitillo