You have three minutes to extract structured data from customer emails. The model hallucinates fields. You try a more detailed prompt. Still hallucinates. You add examples. Better, but not reliable. You walk through the reasoning step-by-step. It works.
The difference between those attempts isn’t luck — it’s three distinct prompting patterns that trade off simplicity for accuracy. Understanding when each one fails matters more than knowing they exist.
Zero-Shot: Speed Over Accuracy
Zero-shot means you give the model a task with no examples. Just the instruction and the input.
# Zero-shot prompt
Extract the customer's order total, product name, and shipping address from this email:
---
[customer email here]
---
Respond in JSON format.This works when: the task is simple, the domain is common (customer service, basic classification), and you care more about speed than precision. Claude Sonnet 4 and GPT-4o handle this well on straightforward tasks — typically 70–85% accuracy on structured extraction if the data is clearly present in the source text.
This fails fast when: the task requires domain knowledge, pattern recognition across edge cases, or multi-step logic. A zero-shot prompt asking a model to classify suspicious financial transactions without examples will miss subtle patterns a few-shot prompt catches immediately. The model has no reference point for what “subtle” looks like in your domain.
Token cost: lowest. You’re sending one instruction and one input. For high-volume tasks (thousands of inferences), this difference compounds.
Few-Shot: The Practical Middle Ground
Few-shot adds 2–5 examples before the actual task. The model learns the pattern from these examples without explicit retraining.
# Zero-shot version (baseline)
Classify this support ticket as urgent or routine:
Ticket: "My payment failed and my order was supposed to arrive today."
# Few-shot version (improved)
Classify this support ticket as urgent or routine.
Examples:
Ticket: "I can't log in to my account."
Classification: Routine
Ticket: "Payment failed and my order was supposed to arrive today."
Classification: Urgent
Ticket: "How do I reset my password?"
Classification: Routine
Ticket: "Server is down and customers can't check their order status."
Classification: Urgent
Now classify this:
Ticket: "My payment failed and my order was supposed to arrive today."
Classification:The few-shot version shows the model exactly what you mean by “urgent” in your specific context. Payment failure + delivery promise = urgent. The model doesn’t guess at definitions — it extrapolates from your examples.
This works when: you have 3–10 clear, representative examples and the task has recognizable patterns. Few-shot consistently outperforms zero-shot by 15–30% on classification and extraction tasks across benchmarks (Anthropic internal testing, January 2025). It’s reliable enough for production use on medium-stakes tasks.
This fails when: your examples don’t cover edge cases, or the pattern is genuinely complex. If your “urgent” ticket examples all involve payment issues but a ticket about data privacy concerns also needs urgent handling, few-shot learns the wrong pattern. It’s only as good as your examples.
Token cost: higher than zero-shot, lower than chain-of-thought. Three examples add ~300–500 tokens per request. At scale, this matters.
Chain-of-Thought: Reasoning Over Pattern Matching
Chain-of-thought asks the model to explain its reasoning before answering. “Think through this step-by-step” is the simplest version. The model shows its work, and accuracy improves — sometimes dramatically.
# Few-shot without reasoning
A customer ordered 3 widgets at $50 each with $15 shipping.
They applied a 10% discount code. What's the final total?
Total: $165
# Few-shot with chain-of-thought
A customer ordered 3 widgets at $50 each with $15 shipping.
They applied a 10% discount code. What's the final total?
Let's work through this step-by-step:
1. Widget cost: 3 × $50 = $150
2. Add shipping: $150 + $15 = $165
3. Apply 10% discount: $165 × 0.90 = $148.50
4. Final total: $148.50
Final answer: $148.50The model gets it right. Without the step-by-step breakdown, it might confuse the order of operations or apply the discount incorrectly. Chain-of-thought forces intermediate reasoning that exposes flawed logic.
This works when: the task involves math, logic, multi-step reasoning, or complex trade-offs. OpenAI’s research shows chain-of-thought can improve accuracy on math and reasoning tasks by 40–60% (Wei et al., 2022). In production at AlgoVesta, we use chain-of-thought for portfolio analysis prompts — the reasoning output also becomes auditable evidence of how a decision was made.
This fails when: you need speed and the task is simple. A classification task (“is this spam?”) doesn’t need chain-of-thought. The added tokens and latency waste time for no accuracy gain. Also, chain-of-thought can amplify hallucinations if the model confidently explains incorrect reasoning. It’s more transparent about being wrong, but still wrong.
Token cost: highest. The step-by-step reasoning output can double or triple token usage compared to zero-shot. At inference scale, this becomes a real cost constraint.
Which One to Use: A Decision Framework
- Zero-shot: Simple classification, straightforward extraction, high-volume tasks where 80% accuracy is acceptable. Go here first — it’s the baseline.
- Few-shot: Medium complexity, domain-specific patterns, 90%+ accuracy needed. Add examples when zero-shot misses edge cases.
- Chain-of-thought: Reasoning required, math involved, multi-step logic, or when you need to audit the reasoning itself. Combine with few-shot examples for best results on complex tasks.
The three aren’t mutually exclusive. Production workflows often use zero-shot as a first pass (speed check), few-shot for cases that fail, and chain-of-thought for high-stakes decisions that need reasoning validation.
Start With Zero-Shot, Add Only If You Need To
Most teams waste tokens by starting with few-shot when zero-shot would work. Test zero-shot first. If accuracy drops below your threshold, add 3–5 examples and re-test. Only move to chain-of-thought if the task genuinely requires reasoning. In real production systems, you’ll use all three — at different stages of the same pipeline.
