Llm

Agent planning (ReAct, Reflexion); tool categories (knowledge, capability, write actions); function calling; multi-agent systems; planning/tool/efficiency failure modes; safety and compound errors; LLM planning debate

AI engineering as a discipline vs ML engineering; three-layer stack (application dev, model dev, infrastructure); product planning; last-mile challenge; use case taxonomy

*AI Engineering* — Chip Huyen

Five-step progressive architecture (context enhancement, guardrails, router/gateway, caching, agents); model drift detection; orchestration frameworks; user feedback systems (explicit vs implicit, edit-as-preference, degenerate feedback loops, sycophancy)

Evaluation methodology: perplexity, pass@k, BLEU/ROUGE, embeddings, AI-as-a-judge, comparative evaluation (Elo/Bradley-Terry, LMSYS Arena); factual consistency, safety, instruction-following; model selection workflow; API vs self-hosting; evaluation pipeline design; benchmark contamination

Data-centric AI; three criteria (quality, coverage, quantity); acquisition and annotation; data synthesis (rule-based, simulation, AI-powered, reverse instruction); model distillation; model collapse; data processing pipeline

When to finetune vs RAG; memory bottleneck; quantisation (FP32→INT4); PEFT; LoRA (low-rank decomposition, rank, alpha, multi-LoRA serving); QLoRA; model merging (task arithmetic, TIES/DARE, frankenmerging); finetuning tactics

What foundation models are; transformer architecture (prefill/decode, KV cache, MoE); training data constraints; Chinchilla scaling law; post-training (SFT, RLHF, DPO)

Inference metrics (TTFT, TPOT, throughput, goodput, MFU, MBU); prefill (compute-bound) vs decode (memory bandwidth-bound); speculative decoding; KV cache management (PagedAttention, FlashAttention, GQA/MQA); continuous batching; prefill-decode decoupling; prompt caching

Sampling strategies (temperature, top-k, top-p); test-time compute; structured outputs; hallucination (self-delusion + knowledge-mismatch hypotheses); inconsistency mitigation

Prompt anatomy; system vs user prompt; chat templates; in-context learning (zero/few-shot, CoT); prompt decomposition; defensive prompt engineering (extraction, jailbreaking, injection, information extraction); three-layer defence (model/prompt/system)

Retrieval-augmented generation; term-based (TF-IDF, BM25) vs embedding-based retrieval; ANN algorithms (LSH, HNSW, FAISS); hybrid search with RRF; chunking; reranking; contextual retrieval; text-to-SQL; memory model