Working paper · author list provisional This is an unrefereed working paper, not a published article, and it has not been submitted. The author list is provisional: confirmation from the WeCanGroup co-authors is still being sought, and the order may change.

Evaluating the Reliability of LLM-Based Document Intelligence for KYC/AML

An Evaluation Framework for an AI-Native Compliance Platform

Joerg Osterrieder, Lennart Baals, Illia Kosterin, Didier Baertschiger, Nathan Douet & Vincent Pignon University of Applied Sciences of the Grisons (FHGR), Chur · WeCanGroup SA, Geneva · Switzerland
Download PDF

Abstract

This is a position paper. It proposes an evaluation framework and reports no empirical results. The framework is developed against a deployed platform, described qualitatively; the study it specifies is planned, not done. No accuracy figure, hallucination rate or reduction percentage appears anywhere in this paper or on this page.

Large language models are being deployed into client onboarding and anti-money-laundering workflows at regulated institutions, private banking among them, where they extract structured facts from identity papers, ownership charts and source-of-wealth narratives. Whether such a system is accurate is measurable and well studied. Whether it is reliable enough to be trusted inside a regulated process is a different question, and it is the one an institution has to answer.

Aggregate extraction accuracy is the wrong primary criterion for a deployment decision. It averages over error types whose consequences differ by orders of magnitude, and it is silent on the property that threshold-routed human review depends on, namely whether confidence scores are calibrated and sharp. We develop a framework around four dimensions that practice conflates, specifying for each what to measure, on what sampling design and against what acceptance criteria.

We then map the dimensions onto the duties they serve under Swiss supervisory, data-protection and anti-money-laundering instruments, and show that the EU AI Act's high-risk regime does not reach this class of system on the present text of Annex III. That absence widens the case for a voluntary framework: the instruments that do bind say what must be managed without saying what must be measured.

Four Dimensions of Reliability

The framework separates four properties that deployment practice tends to collapse into a single accuracy number.

Dimension 1: Extraction accuracy

Necessary and insufficient. Aggregate accuracy averages over error types whose consequences differ by orders of magnitude, and it says nothing about the property that threshold-routed human review actually depends on.

Dimension 2: Confidence calibration

Whether a stated confidence means what it says, and whether it is sharp enough to separate cases near the operating threshold. Measured with binned calibration error, reliability diagrams and the Brier score as a proper scoring rule.

Dimension 3: Hallucination

Treated as a distinct failure class rather than as accuracy shortfall, following the taxonomy in the companion paper: factual, numerical, fabricated and unsupported outputs each behave differently under review.

Dimension 4: Auditability

A property of the retained record, not of the model. Whether the evidence a supervisor would ask for years later actually survives in a form that reconstructs the decision.

The Stratum Routine Monitoring Cannot See

A compliance system that routes low-confidence cases to a human reviewer generates evidence about the cases it escalated. It generates almost none about the cases it did not escalate — and those are precisely where an undetected error becomes a regulatory failure rather than an operational cost.

Routine monitoring cannot observe this stratum, because nothing in the normal workflow ever re-examines a case the system was confident about. The framework therefore treats the non-escalated stratum as a distinct sampling target with its own design, rather than trusting an aggregate that is dominated by cases a human already checked.

This is also where the framework's relationship to conformal prediction is most exposed: conformal guarantees are marginal, holding on average over the whole distribution, where the guarantee a supervisor needs must hold per stratum.

Regulatory and Governance Implications

It is tempting to reach first for the EU AI Act (2024), but on the present text of Annex III its high-risk regime does not reach LLM-based document intelligence used for client onboarding. That is a finding, not a convenience: the instruments that do bind say what must be managed without saying what must be measured.

FINMA's Guidance 08/2024 identifies model risk and expects governance proportionate to it. The revised Swiss Federal Act on Data Protection imposes accuracy duties on personal data. The Anti-Money Laundering Act and the FATF Recommendations behind it require identification of the contracting party and the beneficial owner. Each creates a duty that reliability evidence would discharge; none specifies the measurement.

One governance consequence follows directly from the calibration dimension: reliability evidence expires. Because models are miscalibrated in ways that shift under distribution change, a system calibrated on one quarter's document mix is not thereby calibrated on the next. A reliability finding is a statement about a period, not a permanent property.

What This Paper Is Not

It is not an empirical evaluation. It reports no measured accuracy, no measured hallucination rate and no improvement figure. The corpus and sampling design it specifies are planned; the annotated dataset and validation study they depend on are not yet complete.

It is not a product benchmark, and it does not rank vendors. The platform it is developed against is described qualitatively, to make the framework concrete rather than to make a claim about that platform's performance.

It is not a compliance opinion. The regulatory mapping identifies which duties reliability evidence would serve; it does not advise any institution on whether a given deployment satisfies them.

References

APA style, 30 sources, matching the reference list in the PDF.

  1. AMLA. (n.d.). Federal Act on Combating Money Laundering and Terrorist Financing (Anti-Money Laundering Act, AMLA). Swiss Confederation, SR 955.0, as amended.
  2. Borella, L. (2025, September). Building a shared benchmarking framework for AI in financial services. FINOS (Fintech Open Source Foundation). Retrieved from finos.org (Published 26 September 2025)
  3. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3.
  4. Desai, S., & Durrett, G. (2020). Calibration of pre-trained transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 295–302).
  5. El-Yaniv, R., & Wiener, Y. (2010). On the foundations of noise-free selective classification. Journal of Machine Learning Research, 11, 1605–1641.
  6. EU AI Act. (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union. (European Parliament and Council)
  7. FADP. (2023). Federal Act on Data Protection (FADP). Swiss Confederation, SR 235.1. (Revised version in force from 1 September 2023)
  8. FATF. (2012). International Standards on Combating Money Laundering and the Financing of Terrorism and Proliferation: The FATF Recommendations. Financial Action Task Force, Paris. (Adopted 2012, as amended)
  9. FINMA. (2024). Guidance 08/2024: Governance and risk management when using artificial intelligence. Swiss Financial Market Supervisory Authority (FINMA), Bern.
  10. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., & Crawford, K. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86–92.
  11. Geifman, Y., & El-Yaniv, R. (2017). Selective classification for deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS) (pp. 4878–4887).
  12. Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning (Vol. 70, pp. 1321–1330).
  13. Honovich, O., Aharoni, R., Herzig, J., Taitelbaum, H., Kukliansy, D., Cohen, V., … Matias, Y. (2022). TRUE: Re-evaluating factual consistency evaluation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) (pp. 3905–3920).
  14. Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., … Liu, T. (2024). A survey on hallucination in large language models: Principles, taxonomy, challenges and open questions. ACM Transactions on Information Systems, 43(2), 1–55.
  15. ISO/IEC. (2023). ISO/IEC 42001:2023, Artificial intelligence management system. International Organization for Standardization / International Electrotechnical Commission, Geneva.
  16. Jaume, G., Ekenel, H. K., & Thiran, J.-P. (2019). FUNSD: A dataset for form understanding in noisy scanned documents. In International Conference on Document Analysis and Recognition Workshops (ICDARW) (pp. 1–6).
  17. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., … Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1–38.
  18. Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., et al. (2022). Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221.
  19. Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020). On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) (pp. 1906–1919).
  20. Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., … Gebru, T. (2019). Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*) (pp. 220–229).
  21. Mozannar, H., & Sontag, D. (2020). Consistent estimators for learning to defer to an expert. In Proceedings of the 37th International Conference on Machine Learning (ICML) (Vol. 119, pp. 7076–7087).
  22. Naeini, M. P., Cooper, G., & Hauskrecht, M. (2015). Obtaining well calibrated probabilities using Bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence (pp. 2901–2907).
  23. NIST. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1, U.S. Department of Commerce.
  24. Osterrieder, J., & Baals, L. (2026). When the AI hallucinates, who pays? Mapping LLM hallucination types to legal liability in EU and Swiss financial regulation. FHGR working paper.
  25. Park, S., Shin, S., Lee, B., Lee, J., Surh, J., Seo, M., & Lee, H. (2019). CORD: A consolidated receipt dataset for post-OCR parsing. In Workshop on Document Intelligence at NeurIPS.
  26. Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., … Reitter, D. (2023). Measuring attribution in natural language generation models. Computational Linguistics, 49(4), 777–840.
  27. Shafer, G., & Vovk, V. (2008). A tutorial on conformal prediction. Journal of Machine Learning Research, 9, 371–421.
  28. Shah, R. S., Chawla, K., Eidnani, D., Shah, A., Du, W., Chava, S., … Yang, D. (2022). When FLUE meets FLANG: Benchmarks and large pretrained language model for financial domain. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 2322–2335).
  29. Tibshirani, R. J., Foygel Barber, R., Candès, E. J., & Ramdas, A. (2019). Conformal prediction under covariate shift. In Advances in Neural Information Processing Systems 32 (NeurIPS) (pp. 2526–2536).
  30. Xie, Q., Han, W., Chen, Z., Xiang, R., Zhang, X., He, Y., et al. (2024). FinBen: A holistic financial benchmark for large language models. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024) (pp. 95716–95743).

Download

The full working paper is 33 pages, typeset with the Elsevier article class, with references in APA style.

Download PDF

Companion paper: When the AI Hallucinates, Who Pays? — the hallucination taxonomy and liability mapping this framework's third dimension builds on.