Tag
Distributed Systems
6 posts tagged Distributed Systems.
Ring Attention: What the Near-Infinite Context Paper Actually Says
Extending context beyond what fits on one GPU isn't just a memory problem — it's a communication design problem. Ring Attention sequences the K/V data through a ring of devices and hides the transfers behind computation. Here's what that actually costs in production.
Dapper: What Google's Distributed Tracing Paper Actually Says
Every distributed tracing tool you use — Jaeger, Zipkin, OpenTelemetry — descends from one design decision Google made in 2010: sample at the trace root, not per-span. The paper explains why, and the failure modes it didn't fully solve.
Kafka: What the Original Paper Actually Says
The original Kafka paper from 2011 had no replication. A broker failure made all unconsumed messages permanently unavailable. The paper treats this as a limitation to fix later, not a deal-breaker. Understanding why explains more about Kafka's design philosophy than any architecture diagram.
MapReduce: What the Google Paper Actually Says
The 2004 Google paper that gave us Hadoop — and everything that replaced it — is worth reading not for the map/reduce abstraction itself, but for the fault tolerance model and the straggler insight. The failure modes are still the failure modes.
Pregel: What the Large-Scale Graph Processing Paper Actually Says
PageRank in MapReduce is O(iterations × full dataset reloads). Pregel fixes this by keeping the graph in memory across iterations and replacing disk I/O with message passing. The 'think like a vertex' model is the insight — BSP is the implementation.
Cassandra: What the Paper Actually Says
We had a Cassandra cluster where DELETE operations made reads progressively slower until queries timed out. Adding more disk space made it worse. The root cause is described precisely in the 2009 paper — but only if you understand that Cassandra cannot actually delete data.