Production

Covariate shift/label shift/concept drift/feature change/label schema change taxonomy; degenerate feedback loops (mechanism, detection, correction via randomisation and positional features); detection methods (KS test, two-sample tests, feature validation, prediction monitoring, sliding vs cumulative statistics); proactive design; retraining strategies

*Designing Machine Learning Systems* — Chip Huyen

When to use ML (nine conditions); research vs production differences (silent failure, latency vs throughput, messy data, fairness, interpretability); four system requirements (reliability, scalability, maintainability, adaptability); business vs ML objective alignment; problem framing (task types, decoupling objectives); mind vs data debate

*Reliable Machine Learning* — Chen, Murphy, Parisa, Sculley, Underwood