Data
8 pages
-
Analytical Data Patterns: Data Warehouse, Data Lake, Data Mesh
concept
Data Warehouse, Data Lake, Data Mesh evolution; four Data Mesh principles; DPQ; OLTP vs OLAP; star/snowflake schema; DDD/Data Mesh alignment (OHS as analytical published language, CQRS for projections)
-
Data Decomposition
concept
Data disintegrators/integrators; five-step process; data domains; data sovereignty; connection quotas; polyglot persistence; database type selection; data ownership (single/common/joint); eventual consistency patterns; four distributed data access patterns
-
Dataset Engineering
concept
Data-centric AI; three criteria (quality, coverage, quantity); acquisition and annotation; data synthesis (rule-based, simulation, AI-powered, reverse instruction); model distillation; model collapse; data processing pipeline
-
Evolutionary Database Design
concept
Expand/contract pattern; Flyway/Liquibase; shared DB decomposition; reporting antipattern; Newman's ~12 database decomposition patterns (database view, wrapping service, tracer write, synchronize in application, etc.)
-
Feature Engineering
concept
Feature operations (missing value handling MNAR/MAR/MCAR, scaling, discretisation, categorical encoding/hashing trick, feature crossing, positional embeddings); data leakage causes and detection; feature importance (top 10 = 50% importance); feature generalisation (coverage × distribution overlap); best practices (split by time, scale after split)
-
Pramod Sadalage
author
Co-author of *Software Architecture: The Hard Parts*; data architect at ThoughtWorks; evolutionary database design and NoSQL specialist
-
Software Architecture: The Hard Parts
source
*Software Architecture: The Hard Parts* — Ford, Richards, Sadalage, Dehghani
-
Zhamak Dehghani
author
Co-author of *Software Architecture: The Hard Parts*; creator of Data Mesh; analytical data and domain data ownership advocate