← All tracks

Self-paced course

LLM Inference & Serving

How large language models actually run in production — KV-cache management, batching, speculative decoding, quantization, and the systems work that turns a model into a serveable endpoint.

10 lessons~2h totalFree
Start learning →
0 / 10 lessons complete

Curriculum

  1. EAGLE: Speculative Decoding with Feature-Level Prediction — What the Paper Actually Says12 min read
  2. LLM.int8(): What the 8-bit Matrix Multiplication Paper Actually Says10 min read
  3. Mooncake: What the KV-Cache-Centric Disaggregated Serving Paper Actually Says14 min read
  4. Titans: What the Test-Time Memorization Paper Actually Says9 min read
  5. SGLang and RadixAttention: What the Paper Actually Says11 min read
  6. SARATHI: What the Chunked-Prefill Paper Actually Says11 min read
  7. Mixture of Depths: What the Paper Actually Says14 min read
  8. Mamba: What the Selective State Space Paper Actually Says11 min read
  9. H2O: Heavy-Hitter Oracle for KV Cache Eviction — What the Paper Actually Says14 min read
  10. Ring Attention: What the Near-Infinite Context Paper Actually Says15 min read