← Week 8 Week 10 →

Week 9: Hallucination Prevention

Verification, grounding, and factuality for trustworthy agents

Week 9 of 12

Learning Objectives

Define hallucination, grounding, and FActScore
Explain different hallucination types and their causes
Implement Chain-of-Verification (CoVe) for fact-checking
Decompose claims into atomic facts for verification
Assess factuality using FActScore and similar metrics
Design a multi-layer hallucination prevention pipeline

Topics Covered

Hallucination types (factual, faithfulness, instruction)
Detection approaches (self-consistency, claim decomposition)
Chain-of-Verification (CoVe) methodology
FActScore metric for factuality
Prevention strategies (grounding, multi-agent review)

Resources

Download Slides (PDF)

Jupyter Notebooks

Verification Pipeline

Required Readings

Paper	Authors	Year	Link
Survey of Hallucination in NLG	Ji et al.	2023	arXiv
Chain-of-Verification Reduces Hallucination	Dhuliawala et al.	2023	arXiv
FActScore: Fine-grained Atomic Evaluation	Min et al.	2023	arXiv

Reading Guide: Hallucination Detection and Prevention

3-4 hours CoVe FActScore Hallucination types

Study of Chain-of-Verification, FActScore, and hallucination mitigation

Primary Paper

Chain-of-Verification Reduces Hallucination in Large Language Models
Dhuliawala, S., Komeili, M., Xu, J., et al. (2023)
arXiv arXiv

Secondary Papers

FActScore: Fine-grained Atomic Evaluation of Factual Precision - Min, S., Krishna, K., Lyu, X., et al. (2023) arXiv
Self-Refine: Iterative Refinement with Self-Feedback - Madaan, A., Tandon, N., et al. (2023) arXiv

View Reading Guide Download (MD)

Exercise: Verification Pipeline

100 Points 5-7 hours Advanced

Implement claim verification and hallucination detection

Learning Objectives

Create: Build verification pipelines
Apply: Implement claim decomposition
Evaluate: Detect and mitigate hallucinations

Tasks

Task	Points	Description
Claim Decomposition	30	Break text into atomic claims
Verification System	40	Implement CoVe pipeline
FActScore Evaluation	30	Evaluate factual precision

View Full Exercise Download (MD)

Key Concepts

Hallucination: LLM generates plausible but factually incorrect content. Not random errors - often confidently stated and internally consistent.

Hallucination Types:

Factual: Fabricated facts, non-existent entities, wrong relationships
Faithfulness: Contradicts provided context or source documents
Instruction: Ignores or misinterprets user instructions

Chain-of-Verification (CoVe): Generate initial response, plan verification questions, answer independently, then produce verified response.

FActScore: Fine-grained atomic evaluation - fraction of atomic facts supported by a knowledge source.

Exercise

Build a fact-checking agent using Chain-of-Verification:

Decompose claims into atomic facts (single verifiable statements)
Generate verification questions for each claim
Answer verification questions independently (without seeing original response)
Check consistency between verification answers and original claims
Generate a corrected response based on verification results

Discussion Questions

What types of hallucinations are hardest to detect? Why?
How should agents express uncertainty without becoming overly cautious?
Can verification be done efficiently at scale, or is it too costly?
Why is independent verification more effective than self-correction?
How do you balance the latency cost of verification against accuracy gains?

Additional Resources

Discussion & Questions

Join the Conversation

Have questions about this week's material? Want to discuss concepts with fellow students?

Week 9 Discussion Ask a Question Report an Issue

← Previous Week Course Home Next Week →