Metrics

*Accelerate: The Science of Lean Software and DevOps* — Forsgren, Humble & Kim

Evaluation methodology: perplexity, pass@k, BLEU/ROUGE, embeddings, AI-as-a-judge, comparative evaluation (Elo/Bradley-Terry, LMSYS Arena); factual consistency, safety, instruction-following; model selection workflow; API vs self-hosting; evaluation pipeline design; benchmark contamination

Low-tech collaborative workshop; 10-step process (domain events, commands, policies, aggregates, bounded context candidates); when to use; two-phase facilitation; remote considerations; Rosa: Big Picture EventStorming for domain boundary/KPI mapping + KPI Value Tree; Process Modeling EventStorming for operational value stream analysis

DORA four key metrics (deployment frequency, lead time for changes, change failure rate, time to restore service); empirical basis (4-year, 23K+ respondents, Accelerate); Lean conceptual basis; cluster analysis; no-tradeoff finding; CABs negatively correlated with both metrics; Goodhart's Law warning; pipeline topology variants; instrumentation; MVD; virtuous cycle mechanism; complementary metrics

GQM framework (Basili & Weiss 1984): goal → questions → metrics → data tree; goal statement structure (purpose/object/issue/viewpoint); metric selection criteria (signal strength, cost, cross-question reuse); 9-step workshop; Foo Service case study (rate-limit incident → heartbeat + fail-fast ADR → 10-minute detection before user impact)

*Software Architecture Metrics* — Ciceri et al.