Reducing Hallucinations in RAG Systems

Hallucinations in RAG systems occur when the model generates information not supported by the retrieved context. Common causes:

No RAG system achieves zero hallucinations. The goal is to minimize them and make them detectable.

Better retrieval is the foundation of hallucination reduction:

Better embedding models — Upgrade to high-quality models like text-embedding-3-large, voyage-2, or Cohere embed-v3. Domain-specific embeddings can significantly improve retrieval.

Reranking — Add cross-encoder reranking after initial retrieval. Models like Cohere Rerank or BGE-reranker score relevance more accurately than bi-encoder similarity alone.

Hybrid search — Combine vector similarity with keyword matching (BM25). Catches cases where semantic search misses exact terminology.

Query expansion — Rephrase queries to improve recall. HyDE (Hypothetical Document Embeddings) generates a hypothetical answer to search with.

Evaluate retrieval quality independently from generation. Bad retrieval guarantees bad generation.

Effective prompts constrain the model to retrieved context:

Explicit grounding instructions: "Answer only based on the following documents. If the answer is not in the documents, say 'I don't have information about that.'"

Citation requirements: "Quote the relevant passage that supports your answer. Include the source document name."

Chain-of-thought: "First, identify which documents are relevant to this question. Then, extract the specific information that answers it. Finally, synthesize a response."

Confidence framing: "Rate your confidence in this answer as high, medium, or low based on how directly the sources address the question."

These techniques reduce but don't eliminate hallucinations.

Add verification layers to catch hallucinations:

Citation checking — Programmatically verify that quoted text exists in retrieved documents. Flag responses with invented citations.

Confidence scoring — Ask the model to rate confidence, or use logprobs if available. Low confidence triggers human review or fallback responses.

Consistency checking — Generate multiple responses and check agreement. Inconsistent answers suggest uncertainty.

Fact extraction — Separately extract claims from the response, then verify each against source documents.

Retrieval score thresholds — If top retrieval scores are low, acknowledge uncertainty rather than generating a confident-sounding wrong answer.

Design interfaces that build appropriate trust:

Transparency about limitations builds more trust than pretending perfection. Users who understand the system's limitations use it more effectively.