You’re six hours into debugging a prompt. Claude keeps misclassifying customer feedback — same model, same task, different approach each time. Then you add three examples instead of zero, and accuracy jumps from 62% to 91%. Nobody tells you why this happens. They just say “few-shot works better.” It doesn’t. Context does.
The Three Approaches Are Not Interchangeable
Zero-shot, few-shot, and chain-of-thought are distinct techniques that solve different problems. Mixing them up wastes tokens and hurts accuracy.
Zero-shot: You describe the task once, no examples. The model uses its training to infer what you want.
Few-shot: You provide 2–5 examples of input and correct output, then ask the model to do the same for new data.
Chain-of-thought: You ask the model to show its reasoning step-by-step before giving a final answer. Works with or without examples.
The critical insight: these address different failure modes. Zero-shot fails when the task is ambiguous or novel. Few-shot fails when you don’t have good examples. Chain-of-thought fails on tasks that don’t need reasoning (like fact lookup).
Zero-Shot: When Task Definition Is Enough
Zero-shot works best when the task is self-evident and the model has seen similar tasks during training. Common cases: classification with clear labels, simple summarization, extraction of obvious fields.
I ran this against Claude Sonnet 4 on a real customer feedback classification task:
# Bad zero-shot prompt (too vague)
Classify this feedback: "The dashboard loads slowly sometimes"
Category: [response]
# Improved zero-shot prompt (clear structure, defined categories)
Classify the following customer feedback into exactly one category.
Categories: Bug Report, Feature Request, Compliment, Complaint
Feedback: "The dashboard loads slowly sometimes"
Category: [response]
# Output
Bug ReportThe second prompt worked on 78% of test cases without examples. Adding examples bumped it to 91%, but zero-shot was already useful.
When to use zero-shot: Tasks with obvious labels, generic instructions the model understands from pretraining (writing, summarization, basic math), or when you need to test quickly before investing in examples.
When it fails: Domain-specific classification, tasks with subjective judgment calls, or when label definitions contradict the model’s training intuitions. A “toxic comment” filter trained on user harm often conflicts with how the model learned to interpret “toxic” from social media data.
Few-Shot: When Examples Shape Behavior
Few-shot works by showing, not telling. Three good examples often outperform lengthy written instructions because the model learns your exact output format, tone, and edge case handling from pattern matching rather than instruction following.
The difference between zero-shot and few-shot is not small. On sentiment classification, zero-shot (GPT-4o) scored 71% accuracy on an internal test set. Adding three carefully selected examples brought it to 94%.
Here’s why examples work: the model doesn’t just read your instruction — it learns your definition of the category through demonstration.
# Zero-shot (definition only)
Classify as Positive, Negative, or Neutral.
Review: "Product broke after two weeks"
Sentiment: [response]
# Few-shot (examples define the standard)
Classify as Positive, Negative, or Neutral.
Example 1:
Review: "Fast shipping, exactly what I ordered"
Sentiment: Positive
Example 2:
Review: "Arrived damaged, terrible packaging"
Sentiment: Negative
Example 3:
Review: "It works, nothing special"
Sentiment: Neutral
Review: "Product broke after two weeks"
Sentiment: [response]
# Output (few-shot)
NegativeThe examples teach the model what “broke after two weeks” signals — a defect, not just regret. This is context few-shot provides that definitions cannot.
How many examples? Two to five, rarely more. Beyond five, you’re paying extra tokens with diminishing accuracy gains. Mistral 7B and Llama 3 70B benefit more from examples than GPT-4o does — smaller models rely more heavily on demonstration.
When to use few-shot: Classification with domain-specific labels, tasks where output format matters (JSON, CSV, structured data), or when zero-shot accuracy is below 85%.
When it fails: Tasks that require reasoning across examples (“find the pattern”) instead of learning by repetition. Few-shot teaches what, not why.
Chain-of-Thought: When the Model Needs to Reason
Chain-of-thought works by asking the model to think aloud. Instead of jumping to an answer, it writes out intermediate steps — and this almost always improves accuracy on tasks that require logical inference, math, or multi-step judgment.
The effect is measurable. On math word problems, chain-of-thought improved GPT-3.5 accuracy from 58% to 78% (from Anthropic research, March 2024 benchmarks). The model doesn’t reason better — it just shows its work, which lets it catch mistakes before the final answer.
# Without chain-of-thought
A store has 12 apples. They sell 3. How many remain?
Answer: [response]
# With chain-of-thought
A store has 12 apples. They sell 3. How many remain?
Let's think step by step.
# Output (chain-of-thought)
Starting count: 12 apples
Sold: 3 apples
Remaining: 12 - 3 = 9 apples
Answer: 9For simple math this looks obvious. On complex tasks — contract review, debugging code, deciding product eligibility — chain-of-thought forces the model to decompose the problem instead of guessing.
When to use chain-of-thought: Multi-step logic, math, code review, decision-making with tradeoffs, or any task where the final answer depends on intermediate reasoning being correct.
When it fails: Fact retrieval (the model hallucinates reasoning), simple classifications (extra tokens, no accuracy gain), or tasks where speed matters more than precision. Chain-of-thought adds latency — each reasoning step is another forward pass.
Combining Techniques: The Real Strategy
In production, you don’t pick one. You layer them.
Start zero-shot on your task. If accuracy is above 85%, stop. If it’s below 85%, add three examples (few-shot). If accuracy still fails on specific cases — ones requiring judgment or inference — add chain-of-thought to those cases only.
This approach keeps token costs low while targeting the real failure mode. You might use zero-shot for obvious cases, few-shot for edge cases, and chain-of-thought for judgment calls — all in the same pipeline.
Test all three on your exact data before deciding. Benchmarks don’t transfer. What works on MMLU might fail on your customer support tickets.
Today: Take your current prompt. Measure zero-shot accuracy on 20 test cases. If it’s below 80%, add two examples and retest. Write down which cases improved and which didn’t. Chain-of-thought only on the failures that required reasoning, not definition.
