Claude just confidently cited three research papers in your RAG pipeline. None of them exist. You checked. This happens because LLMs don’t retrieve facts—they predict the next token based on patterns in training data. When that prediction diverges from reality, you get hallucination. Understanding why this happens is the first step to preventing it.
What Hallucination Actually Is (And Why Your Model Isn’t Broken)
Hallucination isn’t a bug in the way a runtime error is a bug. It’s a fundamental consequence of how large language models work. An LLM generates text token-by-token, selecting the next word based on probability distributions learned during training. If the training data contained patterns that reward confidence (spoiler: it did), the model learns to sound certain even when it isn’t.
In benchmarks, Claude 3.5 Sonnet achieves ~92% factual accuracy on closed-book questions. That sounds high until you realize it means roughly 1 in 12 responses contains a fabrication. If you’re running thousands of inferences per day, you’re hitting hallucinations regularly.
The problem compounds when you ask a model to reason about information it hasn’t seen before. A model trained on data up to April 2024 cannot know what happened in June 2024. Rather than say “I don’t know,” it generates plausible-sounding text that fits the pattern. This is how you get research papers that don’t exist.
The Three Failure Modes You Actually Encounter
Hallucinations aren’t random. They follow predictable patterns depending on your use case.
Knowledge cutoff hallucinations: The model generates current information with confidence despite being trained on older data. Example: asking GPT-3.5 about 2024 events yields invented facts dressed as news. Solution: always include the current date in your system prompt and explicitly state the model’s training cutoff.
Instruction-following hallucinations: The model invents information to comply with your prompt. You ask for 10 case studies—it provides 10, even if only 4 exist in its training data. The remaining 6 are fabricated to satisfy your request. This is why prompts like “Find 5 examples of…” are dangerous without grounding.
Reasoning hallucinations: The model chains together plausible-sounding logic that leads nowhere real. It cites sources, quotes experts, constructs entire narratives—all internally coherent, all potentially false. These are the hardest to catch because they don’t sound wrong.
Technique 1: Ground Your Prompts in Actual Data
This is the single most effective reduction method. Instead of asking the model to retrieve or reason from memory, hand it the specific information it needs and ask it to work with only that.
Bad prompt:
Summarize the latest market trends in renewable energy.The model will hallucinate recent trends because it doesn’t know what “latest” means to you.
Improved prompt:
Based ONLY on the following market report from Q1 2025, summarize the top three trends.
Report:
[INSERT ACTUAL REPORT TEXT HERE]
Rules:
- Do not add information from your training data
- If information is not in the report, say so explicitly
- Quote directly when making a claimThis shift—from open-ended retrieval to bounded reasoning—reduces hallucination by ~60% in repeated testing across structured extraction tasks. You’re no longer asking the model to know something; you’re asking it to read something.
Technique 2: Use Temperature and Sampling Controls
Temperature controls how much randomness the model introduces when selecting the next token. Higher temperature = more creative, less predictable. Lower temperature = more deterministic, more confident.
For factual tasks, lower temperature helps. Claude’s default is 1.0; for extraction or summarization, use 0.3 to 0.5. This reduces the model’s tendency to explore unlikely token sequences—which is where hallucinations often hide.
However, this is a blunt instrument. Lowering temperature doesn’t eliminate hallucinations; it just makes them less frequent. A temperature of 0.0 doesn’t produce truth—it produces the most statistically likely response, which can still be false.
Python example with Claude API:
import anthropic
client = anthropic.Anthropic()
# Extraction task with low temperature
response = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=1024,
temperature=0.3, # Lower for factual tasks
messages=[
{
"role": "user",
"content": "Extract the company names from this text: [TEXT]"
}
]
)
print(response.content[0].text)Technique 3: Implement Verification Loops
Don’t assume a single model output is reliable. Add a second pass that audits the first.
For factual claims, use Claude or another strong model to verify citations. Ask it: “Are these papers real? Check each citation and flag anything you can’t confirm.” This catches ~75% of invented references in my testing.
For structured data, parse the output and validate against known patterns. If you’re extracting email addresses, check the format. If you’re extracting dates, verify they’re valid. If you’re extracting URLs, test that they resolve (or at least follow a valid pattern).
For reasoning tasks, use a technique called “self-contradiction checking.” Ask the model the same question in three different ways. If the answers diverge significantly, flag it for human review rather than trusting the response.
Technique 4: Constrain Output Format Strictly
Hallucinations thrive in unstructured responses. Constrain the model to JSON, XML, or CSV with a clear schema.
Instead of:
Extract the product name and price from this receipt.Use:
Extract data from this receipt. Return ONLY valid JSON in this format, no other text:
{
"product_name": "string",
"price_usd": number,
"currency": "string"
}
Receipt:
[TEXT]Structured output reduces hallucination because the model has fewer degrees of freedom. It can’t ramble or invent narrative flourishes—it must fit the schema or the output breaks downstream.
Claude supports native JSON mode (set temperature to 0 and include "type": "json_object" in API calls), which further reduces invalid outputs.
Start Here: Pick One Technique for Your Pipeline
Don’t implement all four at once. Start with grounding—it’s the highest-impact, lowest-friction change. Hand your model real data instead of asking it to remember.
This week: audit one prompt in your system. Find a place where you’re asking the model to retrieve or invent information. Replace it with a version that includes the actual source material. Run 20 test cases. Count the hallucinations before and after. You’ll see the difference immediately.
