When the AI Hallucinates, Who Pays?

Mapping LLM Hallucination Types to Legal Liability in EU and Swiss Financial Regulation

Joerg Osterrieder & Lennart Baals University of Applied Sciences of the Grisons (FHGR), Chur, Switzerland
Download PDF

Abstract

Financial institutions are deploying large language models (LLMs) across compliance-critical functions including client onboarding (KYC/KYB), sanctions screening, beneficial ownership identification, and suspicious activity reporting. These models hallucinate -- they produce outputs that are factually wrong, numerically distorted, entirely fabricated, or plausible but unsupported by source documents. In regulated financial services, such errors are not mere inconveniences; they can trigger anti-money laundering violations, sanctions breaches, data protection infringements, and supervisory enforcement actions. Yet current regulatory frameworks -- the EU AI Act (Regulation 2024/1689), the Swiss Federal Act on Data Protection (FADP, 2023 revision), FINMA Guidance 08/2024, and the Swiss Anti-Money Laundering Act (AMLA) -- address AI-generated inaccuracies only in general terms, without distinguishing hallucination as a distinct failure mode with its own risk profile and liability implications. This paper closes that gap. We develop a domain-specific hallucination taxonomy for financial compliance contexts, distinguishing four types: factual, numerical, fabricated, and unsupported hallucinations. We then construct a mapping framework that connects each hallucination type to specific legal obligations, liability regimes, and enforcement mechanisms across EU and Swiss regulation. Through three detailed case scenarios -- a missed beneficial owner due to factual hallucination, a fabricated regulatory exemption enabling non-compliant onboarding, and a numerical hallucination causing a failed suspicious transaction report -- we demonstrate how hallucination types activate different liability pathways. We conclude with concrete recommendations for compliance AI governance, arguing that regulators should require hallucination-type-specific risk management rather than treating all AI inaccuracies identically.

Key Contributions

This paper makes three contributions to the understanding of AI liability in regulated financial services.

1

Hallucination Taxonomy

A domain-specific taxonomy for financial compliance contexts, distinguishing four types: factual (F), numerical (N), fabricated (B), and unsupported (U) hallucinations.

2

Mapping Framework

Connects each hallucination type to specific legal obligations, liability regimes, and enforcement mechanisms across EU and Swiss regulation.

3

Case Scenarios

Three detailed scenarios demonstrating how different hallucination types activate different liability pathways, producing materially different legal consequences for deployers, providers, and supervisory authorities.

Hallucination Taxonomy

Four hallucination types relevant to financial compliance, each analytically distinct in their evidentiary characteristics, detectability, and regulatory implications.

Type F

Factual Hallucination

A factual hallucination occurs when the model outputs an incorrect fact where a correct fact exists in the source material. The defining characteristic is that the error is verifiable against a ground truth document.

A compliance extraction pipeline processes a Swiss commercial register extract. The document states that the company's registered agent is "Hans Müller, Zug." The LLM extracts "Hans Müller, Zurich."
Type N

Numerical Hallucination

A numerical hallucination occurs when the model outputs an incorrect number, amount, percentage, or quantitative value. Numerical errors warrant separate treatment because they interact with regulatory thresholds that create binary legal consequences.

A transaction monitoring system processes a wire transfer instruction. The document states a transfer amount of CHF 150,000. The LLM extracts CHF 15,000. This order-of-magnitude error places the transaction below the CHF 25,000 threshold that triggers enhanced monitoring.
Type B

Fabricated Hallucination

A fabricated hallucination occurs when the model generates content -- entities, documents, regulations, citations -- that has no basis in either the source documents or external reality. The model does not misread a fact; it invents one.

The compliance AI states: "Under FINMA Circular 2024/7 on Simplified Due Diligence for Digital Asset Holders, entities with verified blockchain-based identity attestations are exempt from standard documentary evidence requirements." No such FINMA Circular exists.
Type U

Unsupported Hallucination

An unsupported hallucination occurs when the model generates an output that may or may not be true but is not supported by the available source documents. The output is an inference, assumption, or extrapolation that the model presents as established fact.

A KYC extraction pipeline processes onboarding documents. The model's output includes: "The client's funds originate from inheritance, consistent with the declared family wealth background." No document in the onboarding package mentions inheritance.

Case Scenarios

Three scenarios demonstrating how the mapping framework operates in practice, each involving a realistic compliance situation and a specific hallucination type.

Scenario A

The Phantom Beneficial Owner

A Swiss private bank onboards a Luxembourg holding company. The LLM extracts the beneficial owner's name as "Pierre Dubois" instead of "Pierre Dubost" -- a one-character substitution -- and hallucinates the nationality from Belgian to French. The PEP screening module finds no match. The bank onboards with standard due diligence instead of the enhanced due diligence required for PEP relationships.

Type F Type U
Scenario B

The Invented Exemption

A Swiss-licensed cryptocurrency exchange queries its LLM-based compliance assistant about due diligence standards. The system fabricates "FINMA Circular 2024/7 on Simplified Due Diligence for Digital Asset Holders" -- complete with plausible numbering, threshold conditions, and specific requirements. The compliance team accepts the guidance. The client is later linked to a ransomware-related money laundering scheme.

Type B
Scenario C

The Decimal Point

A cantonal bank's transaction monitoring system extracts a wire transfer of CHF 125,000 as CHF 12,500 -- a decimal point error reducing the amount by a factor of ten. The hallucinated amount falls below the CHF 25,000 threshold for manual review of transfers to elevated-risk jurisdictions. No suspicious transaction report is filed. MROS later notifies the bank of an investigation.

Type N

Regulatory Coverage

The paper examines five regulatory frameworks and their interaction with each hallucination type.

EU AI Act (Regulation 2024/1689)

Art. 15 requires an "appropriate level of accuracy" for high-risk AI systems. Art. 14 mandates "effective oversight by natural persons," including the ability to correctly interpret outputs and to override them. Art. 9 requires risk management that identifies known and foreseeable risks. Art. 26 establishes deployer obligations including monitoring for risks.

Swiss FADP (2023 revision)

Art. 21 addresses automated individual decision-making: where a decision based solely on automated processing significantly affects a natural person, the data subject must be informed and given the opportunity to request human review. Art. 22 requires data protection impact assessments for automated decision-making deployments.

FINMA Guidance 08/2024

Principle-based supervisory expectations requiring that AI systems are "fit for purpose" -- validated for the specific compliance use case in which they are deployed. Covers model risk, operational risk, and conduct risk. Connects to FINMA Circulars on operational risk and outsourcing.

Swiss Anti-Money Laundering Act (AMLA)

Art. 3 requires identity verification based on documents of evidentiary value. Art. 6 mandates enhanced due diligence for increased-risk relationships. Art. 9 establishes the reporting duty to MROS for suspected proceeds of crime. Art. 37 makes failure to report a criminal offense (up to CHF 500,000 for intentional, CHF 150,000 for negligent violations).

FATF Recommendations

Recommendation 10 (Customer Due Diligence) and Recommendation 20 (Reporting of Suspicious Transactions) establish the international AML/CFT standards that AMLA implements. The 2024 guidance on new technologies acknowledges AI benefits but cautions that "the use of new technologies does not diminish the obligation to conduct adequate due diligence."

Key Conclusion

The question "who pays when the AI hallucinates?" does not have a single answer. The legal consequences depend fundamentally on the type of hallucination -- a distinction that current regulation fails to make.
  • Deployer liability is primary across all hallucination types.
  • Provider liability is activated primarily by fabricated hallucinations (Type B).
  • Human oversight effectiveness degrades from Type F (high detectability) to Type U (low).
  • The EU AI Act's August 2026 deadline for Annex III high-risk systems creates urgency for hallucination-type-specific governance.

Project Connection

"See, e.g., Innosuisse-funded research projects (2026–2027) investigating AI orchestration for Swiss financial compliance automation."

Learn more about the Compliance Copilot project

Download

Full Paper (PDF)

When the AI Hallucinates, Who Pays? Mapping LLM Hallucination Types to Legal Liability in EU and Swiss Financial Regulation

Download PDF

Citation

Osterrieder, J. and Baals, L. (2026). "When the AI Hallucinates, Who Pays? Mapping LLM Hallucination Types to Legal Liability in EU and Swiss Financial Regulation." Working Paper, FHGR.

Keywords: LLM hallucination, financial compliance, EU AI Act, FINMA, liability, KYC, anti-money laundering