hallucination-prevention-toolkit

LLM Hallucination Prevention Toolkit - Python verification tools for preventing AI hallucinations

Information

Property	Value
Language	Python
Stars	0
Forks	0
Watchers	0
Open Issues	0
License	MIT License
Created	2025-12-19
Last Updated	2026-02-19
Last Push	2026-01-02
Contributors	1
Default Branch	master
Visibility	private

Datasets

This repository includes 82 dataset(s):

Dataset	Format	Size

| REVIEW_FINDINGS.json | .json | 22.44 KB |

| data | | 0.0 KB |

| init.py | .py | 0.36 KB |

| base.py | .py | 2.17 KB |

| calculations.py | .py | 2.51 KB |

| statistics.py | .py | 5.22 KB |

| design_thinking_results.json | .json | 21.18 KB |

| data | | 0.0 KB |

| init.py | .py | 0.04 KB |

| verify_calculations.py | .py | 5.66 KB |

| verify_statistics.py | .py | 6.08 KB |

| check_chapter_structure_valid.json | .json | 1.38 KB |

| check_cross_refs_valid.json | .json | 1.36 KB |

| check_equation_labels_valid.json | .json | 1.34 KB |

| check_theorem_refs_valid.json | .json | 1.33 KB |

| verify_arxiv_invalid.json | .json | 1.15 KB |

| verify_arxiv_valid.json | .json | 1.24 KB |

| verify_dois_invalid.json | .json | 1.18 KB |

| verify_dois_resolves.json | .json | 1.21 KB |

| verify_dois_valid_format.json | .json | 1.26 KB |

| verify_github_invalid.json | .json | 1.17 KB |

| verify_github_valid.json | .json | 1.26 KB |

| verify_huggingface_invalid.json | .json | 1.21 KB |

| verify_huggingface_valid.json | .json | 1.22 KB |

| verify_isbn_invalid.json | .json | 1.1 KB |

| verify_isbn_valid.json | .json | 1.15 KB |

| verify_orcid_invalid.json | .json | 1.12 KB |

| verify_orcid_valid.json | .json | 1.19 KB |

| verify_semantic_scholar_search.json | .json | 1.35 KB |

| check_bib_usage_undefined.json | .json | 1.26 KB |

| check_bib_usage_valid.json | .json | 1.26 KB |

| check_chart_deps_valid.json | .json | 1.34 KB |

| check_figure_exists_missing.json | .json | 1.23 KB |

| check_figure_exists_valid.json | .json | 1.28 KB |

| check_latex_syntax_errors.json | .json | 1.27 KB |

| check_latex_syntax_valid.json | .json | 1.26 KB |

| check_notation_consistent.json | .json | 1.34 KB |

| check_slide_refs_broken.json | .json | 1.25 KB |

| check_slide_refs_valid.json | .json | 1.4 KB |

| check_table_syntax_valid.json | .json | 1.33 KB |

| check_undefined_terms_valid.json | .json | 1.32 KB |

| check_url_format_valid.json | .json | 1.24 KB |

| verify_urls_reachable.json | .json | 1.15 KB |

| extract_blockchain_claims.json | .json | 1.38 KB |

| verify_blockchain_dates_correct.json | .json | 1.41 KB |

| verify_eip_exists_valid.json | .json | 1.26 KB |

| verify_market_data_general.json | .json | 1.29 KB |

| verify_protocol_status_active.json | .json | 1.34 KB |

| verify_code_runs_errors.json | .json | 1.27 KB |

| verify_code_runs_valid.json | .json | 1.29 KB |

| verify_imports_invalid.json | .json | 1.24 KB |

| verify_imports_valid.json | .json | 1.31 KB |

| verify_npm_valid.json | .json | 1.24 KB |

| verify_regex_invalid.json | .json | 1.21 KB |

| verify_regex_valid.json | .json | 1.3 KB |

| blockchain_extract_claims.json | .json | 1.25 KB |

| blockchain_verify_dates.json | .json | 1.27 KB |

| code_blocks_catches_errors.json | .json | 1.15 KB |

| code_blocks_valid_python.json | .json | 1.19 KB |

| comprehensive_bib_valid.json | .json | 1.24 KB |

| comprehensive_chart_deps_valid.json | .json | 1.21 KB |

| comprehensive_figures_exist.json | .json | 1.24 KB |

| comprehensive_latex_syntax_valid.json | .json | 1.21 KB |

| comprehensive_notation_valid.json | .json | 1.25 KB |

| comprehensive_tables_valid.json | .json | 1.23 KB |

| comprehensive_terms_defined.json | .json | 1.16 KB |

| comprehensive_urls_format_valid.json | .json | 1.17 KB |

| comprehensive_valid_beamer.json | .json | 1.33 KB |

| error_detection_broken_refs.json | .json | 1.23 KB |

| error_detection_invalid_dois.json | .json | 1.21 KB |

| error_detection_missing_figures.json | .json | 1.25 KB |

| error_detection_syntax_errors.json | .json | 1.22 KB |

| data | | 0.0 KB |

| verify_calculations_2plus2.json | .json | 1.32 KB |

| verify_calculations_sqrt.json | .json | 1.25 KB |

| verify_calculations_wrong.json | .json | 1.24 KB |

| verify_statistics_mean.json | .json | 1.31 KB |

| verify_statistics_wrong.json | .json | 1.22 KB |

| verify_wikipedia.json | .json | 1.27 KB |

| index.json | .json | 23.58 KB |

| test_plan.json | .json | 9.0 KB |

| test_results.json | .json | 23.04 KB |

Reproducibility

This repository includes reproducibility tools:

Python requirements.txt

Status

Issues: Enabled
Wiki: Disabled
Pages: Disabled

README

LLM Hallucination Prevention Toolkit

A comprehensive toolkit with 47 verification tools for preventing and detecting hallucinations when using Large Language Models, specifically Claude.

docs/ - Documentation and guides
preventing_hallucinations.md - Complete guide on hallucination patterns and prevention
tools/ - Verification tools
beamer_validation/ - LaTeX Beamer slide validation (14 tools)
academic_book/ - Springer book chapter validation (11 tools)
generators/ - Content generation helpers (2 tools)
api_verification/ - Standalone API verification (1 tool)
hallucination_checks/ - General verification (18 tools)

Quick Start

# Install dependencies
pip install requests

# Run all tests
cd tools && python test_all_tools.py

# Run offline tests only (fast)
cd tools && python test_all_tools.py --offline

# Validate a Beamer presentation
python tools/beamer_validation/validate_beamer.py slides.tex

# Validate a Springer book chapter
python tools/academic_book/validate_book.py chapter.tex

# Verify Python imports exist
python tools/hallucination_checks/code/verify_imports.py script.py

# Check arXiv paper exists
python tools/hallucination_checks/api/verify_arxiv.py 2301.00001

Available Tools

Beamer Validation (14 tools)

Slide cross-references (check_slide_refs.py --book for multi-chapter)
Figure file existence
BibTeX citation matching
DOI verification
Notation consistency and registry building
URL reachability
LaTeX syntax checking (braces, environments)
Slide overflow detection
Chart dependency validation (.py/.pdf pairs)
Term definition ordering

Academic Book Validation (11 tools)

Environment matching (theorem/proof/definition)
Figure numbering sequences
Theorem/lemma numbering
Cross-reference validation
Notation consistency
Running example continuity
Index/glossary completeness
Section ordering
Bibliography validation
Table formatting (booktabs)

Generators (2 tools)

Missing chart stub generation
Glossary extraction from .tex files

Code Verification (4 tools)

Python import verification (PyPI)
npm package verification
Regex pattern testing
Code block execution

Data Verification (2 tools)

Mathematical calculation verification
Statistical claim verification

API Verification (7 tools)

DOI resolution
GitHub repos/users
arXiv papers
ORCID researcher IDs
Semantic Scholar papers
HuggingFace models/datasets
ISBN validation

Blockchain Verification (5 tools)

Blockchain genesis/fork dates
Ethereum EIP/ERC validation
Protocol status (active/defunct)
Crypto market data verification
Blockchain claim extraction

Fact Verification (1 tool)

Wikipedia article/claim checking

Project Checklist Template

Start new academic projects with a ready-made checklist of issues:

# Clone checklist to a new repository
python tools/scripts/clone_checklist.py Digital-AI-Finance/new-slides

# Clone only specific phases
python tools/scripts/clone_checklist.py Digital-AI-Finance/new-paper --phases setup,content,validation

# Preview without creating
python tools/scripts/clone_checklist.py myorg/myrepo --dry-run

The checklist covers 5 phases with 25 issues: - Setup (5): Folder structure, git, LaTeX template, CLAUDE.md, CI/CD - Content (7): Outline, literature, writing, charts, citations, notation, refs - Validation (6): Hallucination preset, figures, DOIs, cross-refs, syntax, URLs - Review (4): Self-review, formatting, peer review, final compile - Publication (3): Final PDF, archive, Zenodo upload

Documentation

See docs/preventing_hallucinations.md for: - Common hallucination patterns (58 documented) - Prevention strategies - Self-check checklists - Tool usage examples - CLAUDE.md integration

License

MIT

hallucination-prevention-toolkit

Information

Datasets

Reproducibility

Status

README

LLM Hallucination Prevention Toolkit

Contents

Quick Start

Available Tools

Beamer Validation (14 tools)

Academic Book Validation (11 tools)

Generators (2 tools)

Code Verification (4 tools)

Data Verification (2 tools)

API Verification (7 tools)

Blockchain Verification (5 tools)

Fact Verification (1 tool)

Project Checklist Template

Documentation

License