Distributed Systems

Data Warehouse, Data Lake, Data Mesh evolution; four Data Mesh principles; DPQ; OLTP vs OLAP; star/snowflake schema; DDD/Data Mesh alignment (OHS as analytical published language, CQRS for projections)

Gateway taxonomy (enterprise/microservices/mesh), capabilities, pitfalls, history

Test pyramid, consumer-driven contracts, Pact, component/integration/E2E testing; scope vs size distinction, test double fidelity hierarchy, user journey tests, formal verification with TLA+; EDM topology testing, integration testing strategies, event data sourcing

The "-ilities": taxonomy, selection, measurement, governance

Independently deployable artifact; static/dynamic coupling distinction; quantum count by architecture style; micro-frontends (including EDM pairing); 3D dynamic coupling space → 8 saga types

Summary and comparison of all 8 styles; decision guide

All 8 architecture styles rated side by side; decision guide; monolith vs understanding-distributed-systems; partitioning type

Uptime/downtime definition; the nines table (90%–99.999%); techniques for high availability; dependency chaining effects; design-for-production philosophy; ROI of availability investment; MTBF/MTTR/RPO/RTO measurement framework; tyranny of the nines antipattern; Allspaw: MTTR > MTBF; SRE: 100% is always the wrong target

Flow-control mechanism: downstream signals upstream to slow producers rather than overwhelm consumers; bounded queues vs unbounded; backpressure vs load shedding (internal vs boundary); credit-based flow control; pull-based streams; anti-patterns (unbounded queues, retries defeating backpressure)

Unix philosophy, MapReduce, join algorithms, Hadoop vs MPP, Spark/Flink dataflow engines, Pregel graph processing

best-effort, reliable (eager/gossip), total order broadcast; consensus requirement; relationship to CRDTs and replication

Resource partitioning, shuffle sharding, cellular architecture

HTTP caching (Cache-Control, ETag, fresh/stale, immutable static resources); reverse proxies; application-layer: side vs inline cache, LRU eviction, TTL, local vs external cache, thundering herd, cascading failure; 80% hit rate as scalability threshold

CAP theorem and PACELC extension; Kleppmann's critique ("best avoided"); timeliness vs integrity distinction

*Chaos Engineering: System Resiliency in Practice* — Rosenthal & Jones (eds.); complex systems theory, Five Advanced Principles, Game Days, LDFI, Chaos Maturity Model, Continuous Verification, Security Chaos Engineering, organisational/human factors

Closed/open/half-open state machine for downstream resiliency

Four reuse techniques: code replication, shared library, shared service, sidecar; core reuse principle (reuse = abstraction + slow rate of change); versioning discipline; orthogonal coupling

failure taxonomy: hardware, incorrect error handling (2014 study), configuration changes, SPOFs, gray failures, resource leaks, cascading/metastable failures; risk = probability × impact; airline/Black Friday/"Trampled" case studies; building-for-tests vs building-for-production; GC death spiral, queue management (LIFO/CoDel), deadline propagation, latency vs capacity cache

Raft, Paxos, Zab; FLP result; ZooKeeper/etcd; equivalence theorem (linearizable CAS = total order broadcast = locks = uniqueness)

When to use linearizability vs causal vs eventual consistency; timeliness vs integrity distinction; decision guide by scenario (locks, balances, feeds, shopping carts, multi-region); read routing as the practical knob; CAP trap warning

Full spectrum: linearizability, sequential, causal, strong eventual, eventual; COPS causal+ implementation; isolation levels; linearizability ≠ serializability; timeliness vs integrity; safety vs liveness

CDN overlay network; BGP limitations; global DNS LB; IXP placement; edge+intermediary caching layers; DDoS shielding

data plane (on critical path, availability), control plane (off-path, consistency); static stability; scale imbalance solutions (file store buffer, push deltas, hybrid); control theory feedback loop

Coupling taxonomy: structural (Ca/Ce, Martin A/I/D, connascence), operational (Newman's implementation/temporal/deployment/domain types), contract (strict/loose, stamp coupling), integration-style spectrum (File Transfer → Messaging); coupling and quantum boundaries; coordinated deployments as coupling signal; concept leakage and flatten-when-publishing

Conflict-free Replicated Data Types; semilattice + LUB merge; strong eventual consistency; LWW/MV registers; CALM theorem (application-level consistency, not linearizability)

Data disintegrators/integrators; five-step process; data domains; data sovereignty; connection quotas; polyglot persistence; database type selection; data ownership (single/common/joint); eventual consistency patterns; four distributed data access patterns

*Designing Data-Intensive Applications* — Martin Kleppmann

ACID, 2PL, OCC, MVCC, 2PC, Spanner, Saga, Outbox; XA limitations; coordination-avoiding correctness; Newman's "just say no" to 2PC during migration

Hierarchical resolution process; TTL trade-offs; DNS as eventually consistent KV store; DNS as SPOF; static stability principle

*Enterprise Integration Patterns* — Hohpe & Woolf

Broker vs mediator topologies; DDD: three event types (notification, ECST, domain event); private vs public events; distributed big ball of mud anti-pattern; data liberation

Timeouts (fundamental limitation), pings vs heartbeats, when to use active detection; imperfect failure detection theorem

The 8 fallacies (Deutsch/Sun 1994): network reliable, latency zero, bandwidth infinite, network secure, topology fixed, one admin, transport free, homogeneous network; stamp coupling; distributed logging/transactions/contracts

*Foundations of Scalable Systems* — Ian Gorton

*Fundamentals of Software Architecture* — Richards & Ford

HTTP/1.1 vs HTTP/2 vs HTTP/3 (QUIC); multiplexing; HOL blocking; connection management; relationship to REST

Idempotency keys (atomicity requirement, principle of least astonishment), at-least-once delivery, retry safety

the four EIP integration styles (File Transfer, Shared Database, RPI, Messaging); eight decision criteria; trade-offs and when to use which style

Raft state machine (follower/candidate/leader); election terms; CAS+lease practical approach; fencing tokens for mutual exclusion; leader as SPOF

DNS LB, L4 (transport), L7 (application), service discovery, health checks, power of two choices, sidecar as client-side LB; stateless services as prerequisite for scale-out

Physical clock failures (drift, NTP jumps, monotonic); happened-before relation; Lamport clocks (total order); vector clocks (partial order, concurrent detection)

Dynamic configuration (config store, runtime re-read); feature flags (progressive rollout, A/B testing, kill switch, deployment-release decoupling); operational triad: monitor → observe → manage

Author of *Designing Data-Intensive Applications*; distributed systems researcher at Cambridge; CRDT and local-first software advocate

*Mastering API Architecture* — Gough, Bryant, Auburn

Command/Document/Event message types; one-way/req-resp/broadcast styles; Request-Reply (sync block vs async callback; Return Address; Correlation Identifier); point-to-point/pub-sub/datatype channels; Pub-Sub as distributed Observer (push vs pull; channel design); at-least-once delivery; guaranteed delivery; exactly-once via idempotency; invalid vs dead letter channel; backlogs; poison message isolation; competing consumers; data safety trade-off (publisher confirms + persistent queues + manual ACKs); quorum queues (RAFT); RabbitMQ internals; Pipes and Filters; Message Router; Message Bus; Messaging Bridge; Aggregator; Resequencer; Composed Message Processor; Scatter-Gather; Routing Slip; Process Manager; Message Broker (architectural pattern); Envelope Wrapper; Content Enricher; Content Filter; Claim Check; Normalizer; Canonical Data Model; Messaging Gateway; Messaging Mapper; Transactional Client; Polling Consumer; Event-Driven Consumer; Competing Consumers; Message Dispatcher; Selective Consumer; Durable Subscriber; Idempotent Receiver; Service Activator; Control Bus; Detour; Wire Tap; Message History; Message Store; Smart Proxy; Test Message; Channel Purger

Fine-grained domain services; maximum quanta; highest agility/cost; DDD: deep module heuristic, BC vs microservice relationship, subdomain as safe granularity, OHS/ACL for deeper services

Observability as superset of monitoring; three telemetry sources (metrics, logs, traces); structured event logs (one event per work unit, request ID, sampling); distributed tracing (trace ID propagation, spans, Zipkin/X-Ray); metrics and traces as derived views of event logs; Newman: log aggregation first, correlation IDs, Jaeger, synthetic transactions

Decision guide: when to use a central coordinator vs event-driven choreography; team-ownership heuristic; God Orchestrator anti-pattern; workflow state management options

Orchestration-driven SOA; technically partitioned; legacy enterprise pattern

Atomic write + publish without 2PC; relay process; CDC; data liberation patterns; DDD: two wrong approaches + correct approach; NoSQL embedding; pull vs push relay

Key range, hash, consistent hashing, secondary indexes (local/global), rebalancing strategies, request routing; cross-partition complexity costs

Co-author of *Software Architecture: The Hard Parts*; data architect at ThoughtWorks; evolutionary database design and NoSQL specialist

Load shedding (503, priority/age ordering), load leveling (async channel + auto-scaling), rate limiting (sliding window buckets, distributed atomic increment, fail-open), constant work pattern (periodic full-state dump, antifragile, self-healing)

State machine replication (Raft), chain replication (head/tail topology, failure modes, data/control plane split), leader-follower, Dynamo-style; replication lag anomalies; multi-leader conflict resolution

Transient failure recovery; exponential backoff with jitter; retry amplification in chains; idempotency prerequisite; retry queues

Author of *Understanding Distributed Systems*

Distributed transactions via local transactions + compensations; 8 saga type taxonomy; semantic vs implementation coupling; orchestration/choreography trade-offs; DDD: saga vs process manager distinction; stateless/stateful saga; process manager as aggregate; Newman: backward/forward recovery, semantic rollbacks, step reordering, team-based coordination choice

definition; replication vs optimization strategies; scale up vs scale out; stateless services requirement; Amdahl's Law; hyperscale; quality attribute trade-offs (performance, availability, security, manageability); architecture evolution pattern

serverless model (pay-per-invocation, managed autoscaling); cold start by runtime; GAE autoscaling parameters; AWS Lambda (freeze/thaw, provisioned/reserved concurrency, burst limits); parameter study methodology; vendor lock-in

Granularity vs modularity distinction; six disintegrators; four integrators; volatility-based decomposition; MTTS; trade-off-to-business-question method

4–12 coarse domain services; pragmatic middle ground; supports ACID

*Software Architecture Patterns* — Mark Richards

*Software Architecture: The Hard Parts* — Ford, Richards, Sadalage, Dehghani

In-memory replicated cache; extreme performance/scalability; high cost

Log-based brokers (Kafka), stateless topology primitives, partition assignment, windowing, late event strategies, watermarks vs stream time, reprocessing, state stores (internal/external, changelog, hot replicas), effectively once processing, heavyweight vs lightweight framework comparison, stream joins, exactly-once fault tolerance; Kafka production mechanics: producer batching (acks/idempotence), consumer commit semantics, semantic partitioning, consumer groups, ISR

When to use synchronous vs asynchronous communication; temporal coupling as the core distinction; availability chaining risk; async concerns (idempotency, ordering, correlation, observability); mixed model (sync north-south, async east-west); decision guide

Link models (fair-loss/reliable/authenticated), process failure models (Byzantine/crash-recovery/crash-stop), timing models (sync/async/partial sync); default assumptions

Bounding wait time on every blocking call; sizing by P99.9; absent-timeout gotchas; relationship to circuit breaker and retry

TLS encryption (asymmetric key exchange + symmetric data), authentication (certificate chain, root CA), integrity (HMAC), handshake, certificate expiry risk

*Understanding Distributed Systems* — Roberto Vitillo

Co-author of *Software Architecture: The Hard Parts*; creator of Data Mesh; analytical data and domain data ownership advocate