In-context learning is not magic: what the GPT-3 paper actually shows
Reading Brown et al. (arXiv 2020) while debugging prompt sensitivity failures in a production classification pipeline.
The first thing my team did when we got API access to a large language model was throw few-shot examples at it and see what stuck. It worked, so we shipped it. Six months later we had a brittle system that broke whenever the example ordering changed, that performed mysteriously worse on certain query types, and that we couldn't debug because we'd never actually read what the authors said about failure modes.
The GPT-3 paper — Language Models are Few-Shot Learners, Brown et al., arXiv 2020 — is cited in roughly every LLM pitch deck I've ever seen. The citation is almost always to justify "few-shot learning works." The paper also documents the things that don't work, a contamination bug the authors couldn't fix before publication, and benchmark gaps that should make you think carefully before trusting in-context learning on your actual task. Here's what a close read gave me.
What in-context learning actually is
The paper draws a sharp distinction between three paradigms. Most practitioners conflate them.
Fine-tuning updates model weights. You need labeled data, compute, and the result is a new model checkpoint. The weights change. Gradient flows.
In-context learning does nothing to the weights. You construct a prompt — natural language task description, optional demonstrations, then the actual query — and the model processes it in a single forward pass. The authors define it precisely: "the inner loop of this process, which occurs within the forward-pass upon each sequence." No gradient updates. No weight changes. The model is doing something more like pattern-matching on structure in the prompt than anything resembling learning in the training sense.
The three in-context settings the paper evaluates:
- Zero-shot: Only a natural language task description. "Translate English to French:" with no examples.
- One-shot: One demonstration plus the task description.
- Few-shot: Multiple demonstrations, constrained by the context window (2,048 tokens at the time).
These aren't a spectrum you traverse by adding examples until something works. They have meaningfully different reliability profiles and the paper treats them as separate evaluation conditions.
The scaling law that actually matters
The paper reports smooth power-law improvement in language modeling loss across model sizes from 125M to 175B parameters. This gets cited as "bigger is better." The more interesting finding is what scaling does to in-context learning specifically.
On LAMBADA (sentence completion): few-shot accuracy improves from roughly 60% at 13B parameters to 86.4% at 175B. That 26-point gap doesn't appear gradually — the curve is steep at the top.
On arithmetic, there's a cliff. The 13B model achieves around 50% on 2-digit addition. GPT-3 at 175B achieves 100%. For 5-digit operations, GPT-3 drops to 9-10%. The model isn't memorizing arithmetic; it's attempting the computation. The paper notes it "makes mistakes such as not carrying a '1'."
The production implication: the benchmark numbers you see for in-context learning are from a 175B model. If you're working with a smaller model — either for cost, latency, or API availability reasons — the gap between zero-shot and few-shot performance is smaller, and the capability ceiling is lower. The scaling paper (Kaplan et al.) tells you the loss curve. The GPT-3 paper tells you what capabilities emerge at what scale. They're different questions.
Where the model actually fails
This is the section that doesn't make it into pitch decks. The benchmark gaps between GPT-3 few-shot and fine-tuned models are significant for several task classes:
Reading comprehension and structured reasoning:
- RACE: 46.8% vs 90.0% SOTA — a 43-point gap
- DROP (numerical reasoning): 36.5% vs 89.1% SOTA — 52.6 points
- QuAC: 44.3% vs 74.4% SOTA
Natural language inference:
- ANLI Round 3: GPT-3 closes only about 50% of the gap to SOTA
- WiC (word sense across sentence pairs): 49.4% — statistically indistinguishable from random
Two-sentence comparison tasks generally: The paper explicitly calls this out: "GPT-3 appears to be weak in the few-shot or one-shot setting at some tasks that involve comparing two sentences or snippets." Entailment (RTE, CB), semantic similarity, coherence — any task requiring the model to relate two separate pieces of text rather than extend a single passage is a known weak point.
Word manipulation:
- Reversed words: 0.44% accuracy
- Complex anagrams: 15.1% These aren't failure modes that prompting tricks can fix. The model is fundamentally not doing character-level manipulation well.
Translation direction asymmetry: Translation into English substantially outperforms translation from English. The example from the paper: English→Romanian gets 21.0 BLEU while Romanian→English gets 39.5 BLEU. The cause is the tokenizer — GPT-2's byte-level BPE was trained on an almost entirely English corpus, so non-English tokenization is worse.
The data contamination problem they couldn't fix
Section 4.1 of the paper is worth reading carefully. The authors discovered, during analysis, that their training data pipeline had a bug in the overlap filtering logic — relevant test set content wasn't being correctly excluded from the training corpus.
The paper flags affected datasets with asterisks and acknowledges directly: "a bug in the filtering caused us to ignore some overlaps, and due to the cost of training it was not feasible to retrain the model."
LAMBADA has significant contamination. PIQA has potential contamination. The authors run contamination analysis and conclude the effect is "negligible" for most datasets, but the methodology is necessarily imperfect — if you can't identify all the contaminated examples, your contamination estimate has a floor.
The production implication isn't about GPT-3 specifically. It's about benchmark trust. When you evaluate an LLM on a benchmark, you're trusting that the benchmark was held out during training. For models trained on web-scale data, that assumption requires verification, not faith. The GPT-3 authors at least disclosed the problem. Not all model releases do.
The training data is not what you think
The paper reports the Common Crawl corpus at 45TB compressed before filtering, 570GB after. The filtering pipeline does three things: quality filtering (removing content that doesn't resemble high-quality reference corpora), fuzzy deduplication, and blending with known-high-quality sources.
The final training mix (Table 2.2) is not Common Crawl-dominated:
| Dataset | Tokens | Training weight |
|---|---|---|
| Common Crawl (filtered) | 410B | 60% |
| WebText2 | 19B | 22% |
| Books1 + Books2 | 67B | 16% |
| Wikipedia | 3B | 3% |
Wikipedia is 3% of the training mix but sampled 3.4x — seen 3.4 times during training. Common Crawl is 60% of the mix but seen only 0.44 times. The model's "world knowledge" is disproportionately from books and Wikipedia relative to raw token counts.
This matters when you're reasoning about what kind of domain shift to expect. Code, domain-specific jargon, proprietary formats, non-English text — these are all underrepresented. The model's confident output on rare domains is coming from less evidence than its confident output on Wikipedia-adjacent topics.
Production tradeoffs
Prompt sensitivity is not a quirk, it's structural. In-context learning is sensitive to example ordering, example selection, and formatting. This isn't a bug in your implementation — it's a consequence of the mechanism. The model has no way to separate signal from noise in a few-shot prompt the way gradient descent can over thousands of examples. If your production system can't tolerate variance from prompt changes, you need fine-tuning, not better prompt engineering.
Few-shot calibration is inconsistent. The paper notes that GPT-3 "is often not well-calibrated" — its confidence doesn't reliably match its accuracy. Calibration is better than smaller models, but not reliable enough to use confidence as a routing signal without domain-specific validation.
Context window is a hard constraint that degrades gracefully before hitting it. With 2,048 tokens (at publication; modern models have extended this significantly), you can fit roughly 10-100 demonstrations depending on length. More demonstrations generally help, but they also push your query text further from the beginning of the context, and the model's attention is not uniform across the window. Long-context models have partially addressed this, but the fundamental attention distribution issue remains.
The model is doing compression, not lookup. Web-scale training data is compressed into weights. When the model "knows" something, it's reconstructing from a lossy representation. This is fine for common knowledge. For rare facts, specific numbers, recent events, or proprietary information — the compression loses too much. This is what RAG is for.
When not to use in-context learning
If your task requires comparing two pieces of text. Entailment, semantic similarity, coherence scoring — the paper documents this directly. Fine-tune or use embedding-based similarity, don't rely on the model to do this reliably in-context.
If character-level or word-level manipulation is part of the task. Spelling, counting characters, reversals, anagrams — GPT-3 achieves 0.44% on reversed words. This isn't something prompt engineering can fix.
If you need calibrated confidence. Few-shot outputs aren't well-calibrated. If downstream decisions depend on confidence scores, you need either a fine-tuned model with temperature calibration or a different approach entirely.
If your domain is heavily non-English. The tokenizer was trained on English-dominant data. Non-English tokenization is less efficient and the effective context capacity drops. Translation performance is asymmetrically bad for non-English output.
If the benchmark you're using to validate has potential contamination overlap with model training. This applies beyond GPT-3 — any model trained on web-scale data needs explicit held-out evaluation on data you know wasn't in the training set.
If your query volume makes per-call latency meaningful. In-context learning requires the full context — demonstrations plus query — on every call. Fine-tuning pays an upfront cost and then runs inference without demonstrations in the prompt. At high query volume, the token cost of few-shot demonstrations is non-trivial.
What the paper actually established
The useful insight from GPT-3 isn't "175B parameters." It's the demonstration that task generalization can emerge from scale without any task-specific training — and the clear documentation of where that generalization breaks down.
In-context learning is a capabilities probe, not a deployment strategy. You use it to figure out whether a model can do your task at all. If it can, and your task is important enough to invest in, you fine-tune. If it can't, in-context learning won't save you.
The tasks where GPT-3 few-shot still significantly trails fine-tuned models — NLI, DROP, RACE, WiC — aren't easy tasks. They represent categories of reasoning that scale alone doesn't solve. Four years later, with models orders of magnitude larger, some of these gaps have closed. Others have not. Reading the failure modes from the original paper tells you which battles are worth fighting with prompting and which ones need a different weapon.
The benchmark numbers — 86.4% on LAMBADA, 71.2% on TriviaQA — are from a specific model, a specific prompt format, a specific evaluation date. They're not the point. The mechanism and the documented failure modes are.