AI-Native Compliance in Swiss Private Banking
The Wecan Compliance Copilot
Joerg Osterrieder, Lennart Baals and Illia Kosterin, Fachhochschule Graubünden. Didier Baertschiger, Nathan Douet and Vincent Pignon, WeCanGroup SA.
The description of the platform was supplied by WeCanGroup SA. Every evaluative statement, and every statement about what has and has not been measured, is the FHGR authors' alone.
A teaching case for MSc and executive education. One 90-minute session. August 2026, version 1.0.
Innovation project supported by Innosuisse
Geneva, Thursday 10 September 2026
Sylvie Mercier has until Tuesday.
She is head of compliance at a Geneva private bank, and for eleven weeks her team has run the Compliance Copilot in parallel with the process it is meant to replace. The system reads each new client file and extracts the fields that matter. A compliance officer then reads the same file and does the same work by hand. The two are compared. Four hundred files have been through it.
On Tuesday the risk committee wants a recommendation, and the question is narrow. Should the bank stop reviewing every case, and route only the ones where the system is unsure? If so, how unsure is unsure enough?
Her deputy wants 0.90. Anything the system reports at 0.90 confidence or above goes through unreviewed; the rest comes to a human. He points at the pilot data and observes that at 0.90 the system is right about ninety-one percent of the time, and that a compliance officer reviewing everything is a control the bank cannot afford to keep as volumes grow.
Her chief operating officer has done a different sum, and she puts it flatly. "At 0.90 we are still checking three fields in five by hand. That is a hundred and nine hours a quarter and I have seventy. Come down to 0.80 and it fits, at sixty-two. Nobody has told me what that costs and I would like to know before Tuesday."
Mercier has a reason of her own to want this settled, and it is not a good one. She designed the pilot. Four hundred files, one quarter, her own officers as the yardstick, eleven weeks: that was her specification, approved by her, and if the committee decides the evidence is too thin to act on then it is her evidence they are declining. Last autumn she asked the same committee for two additional compliance headcount and was refused on the grounds that the case was not made. She would rather not go back with another case that is not made.
So she has read the pilot data more carefully than either of them, and what she has found is that it does not say what her deputy thinks it says, and does not say what she needs it to say either. She has the vendor's documentation, eleven weeks of her own numbers, and no independent evaluation of the system at all.
On Tuesday she has to say a number out loud.
The opening scene, and only the opening scene, is invented. Sylvie Mercier, her colleagues, the bank and the pilot are written for teaching. The platform described from section 3 onward is real and so is the research partnership; the exhibits carry invented data, as each one states. Nothing in the scene is a claim about any institution.
Abstract
Client onboarding in private banking is widely described as slow, expensive and largely manual, and the partner firm's briefing material describes meaningful automation as rare across most institutions. The authors have not established a source for that characterisation.
This case examines how Wecan, a Swiss RegTech firm, built an AI-native "Compliance Copilot" to automate the client lifecycle: onboarding, document intelligence, screening, risk scoring, periodic review and perpetual KYC. It then examines the harder question the firm ran into, which is the subject of a research partnership with Fachhochschule Graubünden funded by Innosuisse: how do you establish that a language model is reliable enough to be trusted inside a regulated process?
The firm has built the platform and describes it as going into production use. What the case asks is how anyone would know whether it is reliable enough for that.
The five questions
The class opens with these, so read for them rather than reading and then looking. Each is set out in full, with the exhibits it needs, in section 7 at the end.
- Which single number would you ask a vendor for before signing, and what would you still not know?
- Where should the human sit, and at what confidence? Recommend a threshold and defend it.
- The model that made a 2026 decision no longer exists in 2033. Read the record: what survives?
- Extraction is worse on Italian scans than German ones. Deploy, hold, or disclose?
- The efficiency figures arrived before the reliability study. What would you have required?
Exhibits appear where the text first needs them.
1. The setting
Swiss private banking sits under two pressures that pull in opposite directions.
The first is regulatory. Know-your-customer and anti-money-laundering obligations require a bank to establish who its client is, where the money came from, and whether either fact has changed since the last review. The obligation is continuous, not a one-off at account opening, and the evidence has to be auditable years later.
The second is commercial. Onboarding is the client's first experience of the institution, and a slow onboarding loses clients to faster competitors. Prospects drop out during document collection. Relationship managers spend time on data entry that they could spend on clients.
The work sitting between those two pressures is document work. Somebody has to read each document, extract the facts that matter, check them against each other, screen the names involved, and record the result in the core banking system.
That "somebody" has traditionally been a compliance officer with a checklist. The chart below shows where their time on one file goes. Reading and transcription dominate it; the stage that actually needs their training — deciding whether the client's account of their wealth hangs together — is the shortest of the six.
The costs, as the industry describes them
The sector describes onboarding as slow, costly and lightly automated. Specific figures for the elapsed time and the cost per file circulate widely, and this edition does not reproduce them, for the reason set out in section 8.
2. The problem, stated precisely
The problem is not "too much manual work" and the solution is not "automate it", for three reasons.
First, the work is judgement-shaped, not clerical. Deciding whether a source-of-wealth narrative is plausible is not data entry. Exhibit 1 is a specimen; read it and ask what you would have to establish before signing it off. Some of the work genuinely is clerical, and some of it looks clerical while being judgement in disguise, and the second kind does not announce itself.
Exhibit 1: a source-of-wealth narrative
Invented. A specimen of the free-text material the system is asked to make sense of.
The funds derive principally from the sale in 2019 of my late father's logistics business, based in Bergamo, in which I had held a minority interest since 2004. My brother and I sold our holdings to a regional operator; the proceeds were held with an Italian bank until 2023, when I relocated. A smaller part of the assets comes from rental income on two commercial properties in the same province, held through a family company.
What would you have to establish before accepting this? Which parts could a language model check, which parts could it only paraphrase, and which parts need somebody to pick up a telephone?
Second, the cost of error is asymmetric. A missed field costs a follow-up email. A missed sanctions match costs a regulatory finding, and possibly a fine. Exhibit 2 sets out the spread, which runs over several orders of magnitude, and adds the column that matters more than the cost: who would notice. An automation that reduces average handling time while increasing the rate of a rare, expensive error is a bad trade even if it looks good on a dashboard.
Exhibit 2: what the errors actually cost, and who notices
Invented, and the cost column is an order of magnitude rather than a figure.
| Error | Who notices | When | Consequence | Order of cost |
|---|---|---|---|---|
| Missing address line | The client | Within days | A follow-up email | Tens of francs |
| Misread date of birth | Screening, as a mismatch | Within days | A false negative on a name check that should have hit | Hundreds |
| Wrong ownership percentage | Possibly nobody | At the next periodic review, or never | The risk classification rests on a wrong control structure | Thousands |
| Missed politically-exposed-person flag | Nobody, until a supervisor asks | Years later | Enhanced due diligence that was never performed | Supervisory finding |
| Fabricated beneficial owner | Nobody: it passes every plausibility check | Possibly never | The file records a person who does not exist | Supervisory finding, and the record is fiction |
One aggregate accuracy number averages over every row of this table. Two of these rows share an entry in one column, and it is the column that should worry you most. Which two, and which column?
Plotted, the table has a shape. The errors that cost least are caught within days; the ones that cost most are noticed at a later review, when a supervisor asks, or never. The pattern runs diagonally the wrong way, and it is the reason the rest of this case is about evaluation rather than about accuracy.
Third, the failure mode of the obvious approach is well known. Bolting AI onto legacy core banking systems tends to produce hidden costs and limited flexibility rather than genuine automation: an extraction layer that produces output nobody downstream can consume, requiring a human to re-enter the data the machine just read.
3. The organisation
Wecan is a Swiss RegTech firm, meaning it builds technology for regulatory compliance rather than for banking itself. It operates a compliance platform used by private banks and independent asset managers. Its existing product line, as the funding application describes it, is a blockchain-based compliance platform serving Swiss financial institutions.
The firm's strategic position matters for the case. An established client base among Swiss private-banking institutions gives it distribution, and it places the firm inside the industry whose documents the system has to handle rather than outside it.
It also sets up a constraint that runs through the case. A system of this kind can only be evaluated against documents of the sort it meets in production, and obtaining those is a governance problem in its own right: they are client-confidential, they contain personal data, and who may hold them for what purpose is settled in agreements rather than in engineering.
Who carries the consequence
One asymmetry precedes the technology. The vendor sells a system; the bank carries the finding. If an extraction is wrong and a sanctions match is missed, it is the institution that answers to its supervisor, and the institution that cannot point at the vendor's accuracy figure as a defence. So the party with the strongest interest in knowing how reliable the system is has the least access to the evidence, and the party with the evidence has the weaker incentive to publish it. That is a structural feature of the market rather than a claim about any party to it, and it is the standing argument for independent evaluation.
4. The solution: an AI-native platform
Wecan's design choice was to build the compliance workflow around AI rather than adding AI to an existing workflow. The platform is described as LLM-agnostic: the language model is a replaceable component rather than the architecture.
Exhibit 3 traces one client file through the workflow, and Exhibit 4 follows a single file field by field, with the confidence score attached to each extraction and the routing decision it produced.
Exhibit 3: how one client file moves through the platform
Descriptive. The workflow is as the firm describes it; no volumes or rates are implied.
Exhibit 4: one file, field by field
Invented. A single client file at a review threshold of 0.90.
| Field | Extracted value | Confidence | Routed to | Note |
|---|---|---|---|---|
| Surname | MARCHETTI | 0.97 | Auto-accept | Machine-readable zone, clean |
| Given names | GIULIA MARIA | 0.96 | Auto-accept | Machine-readable zone, clean |
| Date of birth | 1971-03-14 | 0.94 | Auto-accept | Machine-readable zone, clean |
| Tax residence | Italy | 0.91 | Auto-accept | Inferred from address, not stated as such |
| Beneficial owner | Giulia Marchetti | 0.71 | Review | One of three names in the registry extract |
| Ownership share | 62 percent | 0.68 | Review | Sums with two other holdings to 104 percent |
| Source of wealth | Sale of family business, 2019 | 0.55 | Review | Free text, no structured field |
Three fields route to a human and four do not. The three that do are the three where the answer is a judgement rather than a reading, which is the argument of section 2 arriving as a routing decision rather than as a claim. Note also that the ownership share sums with the other declared holdings to 104 percent: the system returned a value, with a confidence score, and the arithmetic is still wrong.
The capabilities:
| Capability | What it does |
|---|---|
| Document extraction | Reads the submitted documents and extracts the relevant fields, with a confidence score attached to each extraction |
| Automated checks | Completeness, plausibility and coherence: is anything missing, is it credible, does it agree with the other documents in the file |
| Screening | Sanctions and politically-exposed-person screening against third-party data providers |
| Risk scoring | Produces a risk classification for the relationship |
| Human-in-the-loop validation | Routes cases to a compliance officer for review, according to configurable thresholds |
Two design commitments sit underneath:
- Data sovereignty. The funded architecture places processing on-premise at client sites, and for the institutions in question where the data physically sits is a procurement requirement rather than a preference. The firm describes Swiss hosting as a design principle. The authors have audited no deployment and make no finding about where processing occurs. The teaching point does not depend on this case: where processing happens is something an institution fixes in a contract and then verifies, not a property of the software, and the distinction matters because only one of those two things survives a change of infrastructure.
- Information-security certification. The firm states that the platform holds ISO 27001 certification. The authors have not seen the certificate or its scope statement. Note what such a certificate does and does not say: it covers information-security controls within a defined scope, and says nothing whatever about whether a model's outputs are reliable. Those are different questions.
The architectural bet
The design turns on confidence scoring plus configurable thresholds. It concedes at the outset that the model will be wrong sometimes and builds the workflow around that fact instead of around an accuracy target. The system's design goal is not uniform correctness; it is to act where it is confident and to escalate where it is not.
The bet pays off only if the confidence scores mean something.
5. The AI core, and where it gets difficult
The technical components are, individually, well understood: LLM-based document intelligence, confidence scoring with decision thresholds, human review, integration with third-party data. A short account of how a language model produces an extraction and a confidence score, and of the terms used below, is in the technical note at the end.
What the law does and does not settle
Before the four difficulties, one thing worth fixing in place, because a threshold discussion drifts towards economics if nobody stops it.
Some of this is not tradeable.
- Identification is an obligation, not a rate. The Anti-Money Laundering Act (AMLA, SR 955.0) art. 3 requires an institution to verify the identity of its client on a document of evidentiary value, and art. 4 requires it to identify the beneficial owner. Those are discharged or not discharged. They are not error rates to be traded against reviewer hours.
- Records. AMLA art. 7 requires the underlying documentation to be kept for ten years after the relationship ends, and to be such that a qualified third party can form a reliable judgement about the transactions and relationship.
- Validation. FINMA Guidance 08/2024, on governance and risk management in the use of artificial intelligence, expects a system to be validated for the specific use case it is put to. That is a requirement about evidence, not about accuracy, and it is the closest the Swiss framework comes to the question at issue here.
- Personal data. The revised Federal Act on Data Protection (FADP, SR 235.1) imposes duties on the accuracy of personal data (art. 6) and gives a person subject to a decision taken solely by automated means the right to be heard and to have it reviewed by a natural person (art. 21).
And one absence worth knowing, because it is so often assumed the other way: the EU AI Act (Regulation (EU) 2024/1689) high-risk regime does not on its present text reach this class of system. Annex III lists neither KYC nor anti-money-laundering document processing. Point 5(b) covers creditworthiness assessment, which is a different activity.
The above is the authors' reading of primary instruments, not legal advice, and no author holds a legal qualification. Where a reading is contestable the provision is named so it can be checked.
None of that tells an institution where to put a threshold. What it does is mark out which errors are a cost to be traded and which are a duty not discharged, and any answer to discussion question 2 that does not distinguish the two is incomplete however good its arithmetic.
The four difficulties
The difficulty is in the evaluation, and it has four parts.
Extraction accuracy is necessary and insufficient. You can measure whether the extracted field matches the document. You should. But an aggregate accuracy number tells you almost nothing about deployment risk, because it averages over error types with wildly different consequences. Exhibit 5 shows the same system's accuracy split by document type and language; the aggregate figure sits in the middle of a twenty-three point spread.
Exhibit 5: extraction accuracy by document type and language
Invented data. A pilot of 400 files that did not take place.
| Document type | Language | Fields | Accuracy |
|---|---|---|---|
| Digital PDF | German | 518 | 0.87 |
| Digital PDF | French | 427 | 0.85 |
| Digital PDF | Italian | 231 | 0.82 |
| Photographed scan | German | 616 | 0.79 |
| Photographed scan | French | 553 | 0.76 |
| Photographed scan | Italian | 455 | 0.64 |
| All segments | 2,800 | 0.79 |
Before reading on, settle two things from the table. Where does the aggregate sit relative to the twenty-three point spread, and why is it there rather than in the middle? And who bears the cost of the worst segment, given that nobody chooses which language their national registry prints in?
Calibration is the property the workflow depends on. A confidence score is calibrated if, when the system says it is 70 percent sure, it is right about 70 percent of the time. Calibration is what makes a review threshold mean anything: without it, "send everything below 0.9 to a human" corresponds to no particular error rate, so the threshold is a number rather than a control. Exhibit 6 is a reliability diagram for the pilot system, and it is worth reading with its error bars rather than past them.
Exhibit 6: reported confidence against observed accuracy
Invented data, the same 400 files. Intervals are Wilson score intervals at 95 percent.
| Confidence band | Fields | Claimed | Observed | 95 percent interval | Gap supportable? |
|---|---|---|---|---|---|
| 0.50 to 0.60 | 196 | 0.55 | 0.54 | 0.47 to 0.61 | No |
| 0.60 to 0.70 | 287 | 0.65 | 0.61 | 0.55 to 0.66 | No |
| 0.70 to 0.80 | 441 | 0.75 | 0.68 | 0.64 to 0.72 | Yes |
| 0.80 to 0.90 | 714 | 0.85 | 0.79 | 0.76 to 0.82 | Yes |
| 0.90 to 1.00 | 1,162 | 0.95 | 0.91 | 0.89 to 0.92 | Yes |
| All bands | 2,800 | 0.83 | 0.79 | 0.77 to 0.80 | Yes |
Read the last two columns before the others, and work out what they permit you to say. The system sits below the diagonal in every band, so it looks uniformly overconfident. The intervals decide how much of that appearance survives, and the answer is not all of it.
That is not a defect in the exhibit. It is what eleven weeks of one bank's intake buys, and it lands in a convenient place: the bands whose gaps hold are the three upper ones, which is where any threshold worth setting would sit, and the bands too sparse to speak are the ones that get escalated under every option on the table.
The question the exhibit leaves open is the one it cannot answer. The pilot establishes that the system claims more than it delivers. It does not establish by how much at any particular threshold, because moving the threshold moves which band you are relying on. A calibration curve measured on four hundred files is evidence that something is wrong. It is not yet a control.
Two things this is not. It is not a substitute for accuracy, and the shorthand that calibration beats accuracy does not survive contact with a counterexample: a system that reports 0.79 on every single extraction, and is right 79 percent of the time, is perfectly calibrated and supports no threshold at all. The second clause carries the counterexample, and omitting it is the commonest misstatement of the idea.
What such a system lacks is resolution: its score does not discriminate, taking high values on easy cases and low ones on hard cases. Resolution is what a review threshold actually needs, and it is a separate property from calibration. Neither implies the other. Calibration is measurable, and in the document-understanding literature it is reported considerably less often than accuracy: benchmarks and system papers report an accuracy or F-score as a matter of course, and a calibration error only rarely.
Hallucination is a distinct failure mode, not just another error. An extraction error returns the wrong value from the document. A hallucination returns a value that is not in the document at all: a plausible-looking beneficial owner, a clause that was never written. The second is worse, because the ordinary check, does this look reasonable?, is precisely the check it passes.
Exhibit 7 shows one field failing both ways. The relevant detail is the confidence score: the fabricated answer carries a higher one than the correct answer did.
Exhibit 7: the same field, failing two ways
Invented.
Registry extract reads: Registry extract lists: Giulia Marchetti 62%, Marco Bellini 24%, Fondazione Alpe 14%
| Case | What the system returned | Confidence | Why it matters |
|---|---|---|---|
| Correct extraction | Giulia Marchetti, 62 percent | 0.71 | Right. Low confidence because three names competed |
| Extraction error | Giulia Marchetti, 26 percent | 0.69 | Wrong, but every token is traceable to the document. A reviewer comparing against the source finds it |
| Hallucination | Marco Rossi, 100 percent | 0.93 | No such name in the document. Confidence is HIGHER than the correct answer, and it passes the check 'does this look like a beneficial owner entry' |
Compare the two failures on one axis only: which would a reviewer catch, and by what method? Then look again at the confidence column.
These are general properties of language models rather than observed failures of any particular system. No independent published hallucination measurement exists for commercial KYC extraction, so no comparison between this platform and any alternative is implied. The claim concerns the state of the evidence, not the standing of any product.
Auditability is a requirement on the record, not on the model. A regulator asking why a relationship was classified as low-risk in 2026 may need an answer well into the following decade. The Anti-Money Laundering Act requires the documentation underlying a client relationship to be retained for ten years after the relationship ends, which for a client who stays means indefinitely. That constrains what the system must retain, and it constrains model choice, because a component that cannot explain its output is expensive to deploy in this setting regardless of how accurate it is. It also raises a harder question than storage. If the model version that made a 2026 decision has been retired by the time the question is asked, what exactly does the retained record establish?
Exhibit 8 is the record for one such decision, as it would stand in 2033.
Exhibit 8: the audit record for one decision
Invented, and built around the same client file as Exhibit 4 so the two can be read together.
| What the record holds | Value | Available when a supervisor asks in 2033? |
|---|---|---|
| Decision reference | KYC-2026-0914-0271 | Yes |
| Date of decision | 14 September 2026 | Yes |
| Field | Beneficial owner | Yes |
| Value written to the record | Giulia Marchetti | Yes |
| Confidence the system reported | 0.71 | Yes |
| Threshold in force at the time | 0.90 | Only if threshold changes are versioned. Many systems store the current value |
| Routing outcome | Escalated: below threshold | Yes |
| Reviewer and outcome | Officer 14, confirmed as extracted | Yes |
| Source span | Registry extract, page 2, lines 4 to 6 | Yes, if the document is retained |
| Model identifier and version | Vendor API, doc-extract, version 2026-03 | Version retired November 2027. Not reproducible. |
| Prompt and configuration | Hash a41f9c2e | The hash, yes. The prompt itself belongs to the vendor |
| Screening list version | Sanctions list 2026-09-12 | Only if list snapshots are retained, which is uncommon |
| Risk classification produced | Low | Yes |
Read the third column, then answer the question the supervisor actually asked: why was this relationship classified as low risk? Which rows help, which do not, and does the set of rows that survive add up to an answer or only to an account?
The stratum nobody sees
One consequence of the threshold design lies beyond the reach of routine monitoring. The cases the system escalates get a second pair of eyes, so errors there are caught and counted. The cases it accepts without review get nothing, so errors there are neither caught nor counted, and they are the ones that reach the record. A dashboard built from reviewer corrections is therefore a dashboard of the errors the system already knew it might be making. The rate that matters most is the one nobody is looking at, and measuring it means deliberately auditing a sample of cases that were never escalated.
Where the evidence would come from
These four problems are the subject of a research collaboration between the firm and Fachhochschule Graubünden, funded by Innosuisse as the innovation project "A specialized and secure AI orchestrator". Its scope covers data quality, reliability, and the evaluation of AI for compliance work. The validation study that would settle these questions depends on an annotated dataset scheduled ahead of it, and neither exists yet.
The system therefore enters production while the reliability question remains open.
6. Reported outcomes
The firm reports substantial reductions in onboarding workload and in unit cost per file, on the basis of internal client estimates. This edition does not reproduce the figures; section 8 gives the reason.
The methodological point stands without any figure: a workload reduction and a unit cost reduction are different quantities, and neither is an accuracy claim. A system can reduce cost per file while making more of a particular kind of error. Efficiency metrics and reliability metrics answer different questions, and the case's central tension is that the efficiency numbers arrived first.
The decision in front of Mercier
Exhibit 9 is the arithmetic she has to defend on Tuesday.
Exhibit 9: the threshold decision
Computed from Exhibit 6. A threshold of t means: accept without review when confidence is at least t, otherwise escalate. Reviewer time is stipulated at 4 minutes per escalated field, not per file: a reviewer re-checks the flagged field against the source, not the whole client file.
| Threshold | Escalated | Reviewer hours | Errors: machine | Errors: reviewer | Total on the record | Of which screening-relevant | Weighted cost |
|---|---|---|---|---|---|---|---|
| 0.50 | 0 | 0 | 598 | 0 | 598 | 256 | 13,158 |
| 0.60 | 196 | 13 | 508 | 20 | 527 | 200 | 10,321 |
| 0.70 | 483 | 32 | 396 | 48 | 444 | 146 | 7,587 |
| 0.80 | 924 | 62 | 255 | 92 | 347 | 102 | 5,351 |
| 0.90 | 1,638 | 109 (over capacity) | 105 | 164 | 268 | 82 | 4,310 |
| 1.00 (dominated) | 2,800 | 187 (over capacity) | 0 | 280 | 280 | 84 | 4,379 |
Mercier has seventy reviewer hours per four hundred files. No row satisfies both her deputy and her chief operating officer, and no arithmetic will make one appear.
The last row is the one to sit with, and it is worth working out before reading the explanation. Escalating every field puts more errors on the record than escalating most of them, and costs seventy-eight extra hours to do it. Nothing is broken; the answer is in the exhibit's own stated assumptions, and finding it is the exercise.
Two things follow. The instinct that a human in the loop is always a safeguard is wrong as stated: a human in the loop is a safeguard exactly where the human is better, and nobody in this case has measured where that is. And a threshold can be too low, which the arithmetic could not previously express.
Exhibit 10 separates two questions that get run together whenever a number like this is published: whether it identifies anybody, and whether it is true. Passing one gate is routinely mistaken for passing both.
Exhibit 10: two gates
Conceptual. Anonymising a number and confirming a number are independent operations, and passing one is routinely mistaken for passing both.
7. Questions to solve
Answer these from the case and its exhibits. Worked answers are in the instructor material, and the honest advice is to form your own first.
- How should the reliability of an LLM that extracts KYC data be evaluated? The vendor will give you one number before you sign. Which number do you ask for, and what will you still not know? Which measurements would you require before deployment, and which would you accept measuring afterwards? Exhibits 5 and 6 are the evidence Mercier actually has. Read Exhibit 6's intervals before you answer.
- Where should the human sit in the loop, and how would you measure the value of putting them there? Using Exhibit 9, recommend a threshold and defend it. Note that the deputy's threshold and the COO's capacity, as they state them in the opening scene, cannot both be satisfied. What would you measure to know you had set it wrong?
- How do you ensure auditability and regulatory acceptance of AI-assisted decisions? Work from Exhibit 8, which is the record for one decision as it would stand in 2033. The model version that produced it was retired in 2027. Go row by row: which entries still answer a supervisor's question, which have quietly become unverifiable, and what would you have had to retain in 2026 to close the gap? Then the harder half: if the decision cannot be reproduced, does a complete record of its stated basis satisfy AMLA art. 7, or only look as though it does?
- How do data quality and bias propagate through a compliance pipeline? Exhibit 5 shows materially worse extraction on Italian-language scans than on German digital documents. You may hold deployment for that segment, deploy with a segment-specific threshold, or deploy unchanged and disclose the disparity to the bank. Choose, and say what you would tell the client whose file was wrongly refused.
- A question about this case rather than about the technology. The firm's efficiency figures arrived before the reliability study was complete, and this edition withholds them because their provenance is not established. Was that the right call? Consider the competing readings: that withholding an unverified number is elementary hygiene, or that removing it withholds context a reader needs and locates the judgement with the authors rather than the reader. If you were the research partner, what would you require before putting those figures in a peer-reviewed paper with your name on it, and what would you do if the confirmation you needed were slow to arrive?
8. Instructor material, and a note on the figures
On the figures in this case
This edition reproduces no performance figures attributed to the firm. Every such figure available to the authors is an unconfirmed estimate or an illustrative one supplied by the firm, none has been independently verified, and the validation study that would settle the question is the research project's next substantial output.
The firm supplied those figures with the caveat already attached: its own case description states that they come from internal client estimates and should be confirmed and anonymised before academic use. The caution here follows the firm's. A figure in that condition can illustrate a point in a seminar, where somebody is present to frame it; on an open page it would travel without them.
The exhibits are a separate matter and are invented outright. Every exhibit describes a pilot that did not happen, at a bank that does not exist, or a document nobody filed. They are stipulations written so the arithmetic in the discussion questions works, which is the ordinary teaching-case device: a stipulation asserts nothing about the world.
Exhibit 8 is computed from Exhibit 6 rather than typed, and Exhibit 5's accuracies are derived from Exhibit 6's, so the three cannot disagree. The unit throughout is the extracted field, not the file: 2,800 extractions across 400 files. That is the first question to ask of any reported accuracy: a file has no single confidence score unless a rule for combining its fields has been chosen, and that choice is rarely stated and never neutral.
Instructor material
The full instructor material is published, at teaching-note.html: learning objectives, a session plan for one 90-minute class with two shorter variants, a board plan, guidance on where the discussion predictably goes wrong, and worked answers to all five discussion questions.
It is published rather than held back. The usual argument for withholding is that a teaching note students can read is not a teaching note; against that, this case has no verdict to spoil. Its answers are arguments rather than solutions, several of them turn on judgements the case deliberately leaves open, and an instructor preparing an unfamiliar case at short notice is better served than a student is disadvantaged. If you are a student and your instructor has not sent you here, the honest advice is to form your own answer to the five questions first. Reading the guidance beforehand will cost you the part of the session worth having.
Technical note
A short account of the machinery, for readers meeting it for the first time. Nothing here is specific to the platform in the case.
How an extraction gets made. A language model reads the document as a sequence of tokens and produces the requested field as another sequence. Modern document-intelligence systems attend over the document at more than one level, roughly words within a region and regions within a page, which is what lets a model find a name on a passport without being told where to look. What matters for this case is the output: a value, and a number attached to it.
What the confidence score is, and is not. It is usually derived from the probabilities the model assigns to the tokens it emitted. Those probabilities are not, in general, the probability that the answer is correct, and treating them as if they were is the most common mistake in this area. Getting from one to the other means fitting a calibrator on held-out data, and then validating the calibrator, which is a second modelling problem rather than a setting to switch on.
Calibration, resolution, and the word people reach for by mistake. A score is calibrated if outcomes match the stated confidence over many cases: of the cases labelled 0.80, about eighty percent are right. A score has resolution if it discriminates, taking high values on easy cases and low ones on hard cases. Both are needed and neither implies the other. A system that outputs 0.86 on every case, when its overall accuracy is 0.86, is perfectly calibrated and perfectly useless, because no threshold divides its output into cases worth reviewing and cases not.
The word usually reached for here is sharpness, and it means something narrower: how concentrated the scores are, judged without looking at outcomes at all. A system emitting 0.99 and 0.01 at random is maximally sharp and has no resolution whatever. Sharpness is necessary for resolution and not sufficient for it, and a review threshold needs resolution. The distinction is worth keeping straight because the literature does.
Precision and recall. For a field the system either fills or leaves blank, precision is the share of the values it filled in that are correct, and recall is the share of the values it should have found that it did find. They trade off: a system can be made more cautious, raising precision and lowering recall, or more eager, with the reverse effect. Neither is more fundamental; which one you care about depends on the cost of the two mistakes, which in this domain differ by orders of magnitude.
Selective prediction. A system permitted to answer "I am not sure" can be far more accurate on the cases it does answer. Plotting accuracy against the share of cases answered gives a risk-coverage curve, and choosing a review threshold is choosing a point on it. This is why calibration is the property the workflow depends on: the curve is only meaningful if the confidence ordering is trustworthy.
Hallucination against extraction error. An extraction error takes a value from the document and gets it wrong. A hallucination supplies a value the document does not contain. The two need separate counting because they need separate remedies: better reading fixes the first, and grounding, meaning a requirement that every emitted value be traceable to a span of the source, is what addresses the second.
Glossary
| Term | In this case |
|---|---|
| RegTech | Technology built for regulatory compliance rather than for banking itself |
| KYC | Know your customer. The duty to establish who a client is and where their money came from |
| Perpetual KYC | Keeping that picture current continuously, rather than re-checking it on a fixed cycle |
| AML | Anti-money-laundering. The regime the KYC duties sit inside |
| Beneficial owner | The natural person who ultimately owns or controls a client entity, however many layers sit in between |
| Source of wealth | An account, often narrative, of how a client's money was accumulated. Assessing it is judgement, not data entry |
| PEP | Politically exposed person. Not prohibited, but a category requiring enhanced scrutiny, because the corruption risk is higher |
| Sanctions screening | Checking names against published sanctions lists, usually supplied by a commercial data provider |
| Core banking system | The bank's system of record. If an extraction cannot be written into it, the automation has not saved anyone any work |
| ISO 27001 | A certification of information-security management within a declared scope. It says nothing about whether a model's outputs are correct |
| Annotated dataset | Documents with the correct answers recorded by hand, so a system's output can be scored. Building one is most of the cost of an evaluation |
| Confidence score | The number the system attaches to an extraction. See the technical note on what it is not |
This case was prepared by Joerg Osterrieder, Lennart Baals and Illia Kosterin of Fachhochschule Graubünden, with Didier Baertschiger, Nathan Douet and Vincent Pignon of WeCanGroup SA, from a case description supplied by the firm in July 2026 and from the signed Innosuisse funding agreement. Teaching cases are developed solely as the basis for class discussion and are not intended to serve as endorsements, sources of primary data, or illustrations of effective or ineffective management. The opening scene and Exhibits 2, 3 and 4 are invented. No figure attributed to the firm or the sector is reproduced in this edition.
© 2026 Fachhochschule Graubünden. Free to use and adapt for teaching with attribution. For other uses, contact the corresponding author.