AI Governance in Financial Services — Gap Series | 9 Gaps. 21 Days. One Questionnaire.
Gap F-G8 of 9 · ISO 42001 Cl. 8.4 / Annex A.6 · Human Oversight
Reading It Once Is Not a Review
The disclosure clause promised clients that a human had reviewed every AI output before it reached them. Marcus asked Jade what that review looked like. The answer was three words. Three words that collapsed the promise entirely.
Series
Finance AI Governance
Framework
ISO/IEC 42001:2023 Cl. 8.4 / Annex A.6
FCA Parallel
SM&CR / Consumer Duty Outcomes
Fix Document
PROC-AIMS-FIN-HITL-001
Severity
Critical
Harcastle Questionnaire — Progress
1
2
3
4
5
6
7
8
9
DOC-AIMS-FIN-DISC-001 was filed. The disclosure framework was built. Question 7: provisionally satisfied. Marcus had moved quickly — but the answer he had given Harcastle contained a statement he had not yet verified.
The disclosure clause said, in plain text: "All AI-assisted content was reviewed and approved by the named Relationship Manager prior to issue."
On Day 14 he called Jade again. Not about the data incident. Not about the ChatGPT suspension. About the review itself.
"When you used ChatGPT for the Harcastle commentary — walk me through what you did before you sent it."
"I read it through. Made sure it sounded right. Then sent it."
A pause.
"That's it?"
"That's what review means, doesn't it?"
Marcus opened the Harcastle questionnaire. Question 8.
Harcastle Group plc — AI Governance Due Diligence
Question 8 of 9
"What human oversight controls exist for AI-generated outputs before they are acted upon or delivered to clients? Please describe the review process, who is responsible, and how oversight is evidenced."
Include details of any checklist, sign-off procedure, or audit trail that confirms human review has occurred.
Harcastle were asking for three things: a process, an owner, and evidence. Whitmore had none of them. What they had was Jade reading AI output once, deciding it sounded right, and sending it. That was the entirety of their human oversight programme.
⚠ Warning — SM&CR and Consumer Duty Exposure
Under SM&CR, Senior Managers are personally accountable for the areas of the firm they are responsible for. Where AI systems are used in client-facing processes, the Senior Manager responsible cannot discharge that accountability by pointing to a general sense that "someone reviewed it." ISO 42001 and the FCA both expect documented, evidenced oversight — not an assumption that reading once constitutes control.
Tracing the Non-Process: What Actually Happened
Marcus spent the afternoon of Day 14 reconstructing exactly how the Harcastle Q3 report had moved from ChatGPT to the client's inbox. What he found was not a review process. It was a sequence of unchecked steps — each one reasonable in isolation, collectively amounting to no oversight at all.
Actual Review Process — Harcastle Q3 Report (Reconstructed)
①
Jade drafts the narrative in ChatGPT. She provides portfolio data, asks for commentary on ESG screening and performance. ChatGPT produces five sections. She copies them into the report template. No record of prompts used. No record of which output version was accepted.
②
Jade reads the output. She checks the flow and the numbers. The SFDR Article 8 reference reads confidently. She does not verify it against the current regulatory text — she has no reason to assume it is wrong. No factual verification step. No regulatory cross-reference check.
③
Jade sends the report. Her name and signature appear. No countersignature. No compliance sign-off. No record that anyone other than Jade saw the document before it left the firm. No second-reviewer requirement. No audit trail entry.
④
Harcastle receives the report. They read the SFDR Article 8 commentary. They rely on it. They have no way of knowing the source. The marginal inaccuracy in the regulatory reference is embedded in their files.
The problem was not Jade. She was not cutting corners — she was operating in a firm that had never defined what "reviewing AI output" meant. Without a procedure, every individual makes their own judgment about what is enough. Sometimes that judgment is right. Sometimes — as with the SFDR reference — it is not.
⚡ Caution — The Confident Inaccuracy Problem
AI language models do not flag uncertainty the way a human expert does. A qualified analyst who is unsure of a regulatory threshold will say so. An AI system will state the threshold with the same confident tone regardless of its accuracy. Human oversight must be specifically designed to catch this — which means reviewers need to know what to check, not just that they should check.
What ISO 42001 Clause 8.4 and Annex A.6 Require
Clause 8.4 of ISO/IEC 42001:2023 establishes the operational requirements for AI system use. It requires organisations to define and implement controls that ensure AI outputs are used appropriately — including controls over how outputs are reviewed before being acted upon. Annex A.6 extends this into specific human oversight controls.
Requirement
What It Means
Obligation
Whitmore — Prior State
Cl. 8.4 — Operational controls for AI use
Documented procedures must exist for how AI outputs are reviewed, validated, and approved before use
SHALL
None
A.6.1 — Human oversight of AI systems
A responsible human must be identified for each AI-assisted process; their oversight role must be defined
SHALL
Implied (Jade) but not documented
A.6.2 — Review criteria defined
The criteria a reviewer uses to accept or reject AI output must be documented — not left to individual judgment
SHALL
None
A.6.3 — Oversight evidenced
A record must exist confirming that human review took place — including who reviewed, when, and what was checked
SHALL
None
A.6.4 — Override capability
Humans must be able to override, reject, or escalate AI outputs; this capability must be part of the documented process
SHOULD
Possible in practice but never invoked
Whitmore failed four of five requirements. The fifth — override capability — existed in theory, but because no process existed, it had never been used in any documented way.
The Fix: PROC-AIMS-FIN-HITL-001
Marcus built the Human-in-the-Loop Oversight Procedure across Days 15 and 16. The procedure defines four review stages — each mapped to a specific AI system type and output category. It is not a single universal checklist. The things a reviewer must verify about an AI-drafted portfolio commentary are different from what they must verify about an automated AML screening result.
Which AI system produced this output? (Confirm AIMS-SYS reference)
✓
What input data or prompt was used? Is a record retained?
✓
Is this the output version I am reviewing — or has it been edited since generation?
2
Factual Verification
During content review
✓
Are all regulatory references verified against current primary sources?
✓
Are all client-specific figures (returns, allocations, dates) confirmed against source data?
✓
Does any statement require professional judgment that the AI cannot reliably provide?
3
Suitability & Fairness
Before approval
✓
Is the output appropriate for this specific client's circumstances, objectives, and risk profile?
✓
Could any part of this output mislead the client or omit material information?
✓
Does the output comply with COBS 4.2 — fair, clear, and not misleading?
4
Sign-Off & Record
Before sending
✓
Reviewer name, role, and date recorded in the AI Output Review Log
✓
AI disclosure clause confirmed present and accurate before send
✓
Any override, amendment, or escalation decision documented with rationale
PROC-AIMS-FIN-HITL-001 — Four-Stage Review Gate. All four stages required for client-facing AI outputs. Stages 1–3 required for internal AI-assisted compliance outputs. Review record retained in AI Output Review Log (REG-AIMS-FIN-OUT-001).
ℹ Note — Differentiating by AI System Type
The four-stage gate applies universally, but Stage 2 (Factual Verification) varies by system. For ChatGPT and MS Copilot (language generation): regulatory references and market data must be cross-checked against primary sources. For LexisNexis AML AI: automated screening results must be reviewed for false positives and false negatives before any adverse action is taken. For Experian Credit AI: credit risk outputs must be reviewed for bias indicators and verified against the firm's own credit criteria before client decisions are made.
What "Oversight Evidenced" Actually Means
Harcastle's questionnaire asked not just whether a process existed — but how oversight was evidenced. This is the part of the question that catches firms most off guard, because many compliance programmes confuse having a procedure with being able to prove the procedure was followed.
PROC-AIMS-FIN-HITL-001 introduces a single evidence artefact: the AI Output Review Log, referenced as REG-AIMS-FIN-OUT-001. Every client-facing AI-assisted output generates one entry in that log. The entry records: the AIMS-SYS system used, the output reference, the reviewer name and role, the date, the four-stage gate completion status, and any amendments made prior to sending.
The log is retained for seven years — consistent with FCA record-keeping obligations under SYSC 9.1 and MiFID II. It is the difference between a compliance programme that exists on paper and one that can withstand a regulatory inspection or a client subject access request.
⚙ ITIL 4 — Service Validation and Testing
The four-stage review gate is a quality assurance practice embedded into the AI output delivery workflow. ITIL 4's Service Validation and Testing practice provides the operational framework: defining acceptance criteria before output is generated, building verification into the process rather than treating it as an afterthought, and capturing evidence that validates the output meets those criteria. Human oversight of AI is not a compliance bolt-on — it is a service quality control.
Marcus's Answer to Question 8
Sub-question
Marcus's Response
Evidence Document
What human oversight controls exist?
PROC-AIMS-FIN-HITL-001 — a four-stage review gate applied to all AI-assisted client outputs, covering provenance, factual verification, suitability, and sign-off.
PROC-AIMS-FIN-HITL-001 v1.0
Who is responsible?
The named Relationship Manager completes Stages 1–3. Stage 4 sign-off requires countersignature by the Head of Compliance (SMF16) for all outputs above the risk threshold defined in REG-AIMS-FIN-RISK-001.
RACI-AIMS-FIN-001
How is oversight evidenced?
AI Output Review Log (REG-AIMS-FIN-OUT-001) — one entry per client output, recording reviewer, date, gate completion, and amendments. Retained seven years.
REG-AIMS-FIN-OUT-001
F-G8
✅ Provisionally Satisfied
PROC-AIMS-FIN-HITL-001 v1.0
REG-AIMS-FIN-OUT-001 (AI Output Review Log)
5 of 21
Marcus filed the procedure at 6:14pm on Day 16. Eight questions answered. One remaining. Five days on the clock.
He sent Diana the progress summary. She replied in four minutes — which, in Diana's communication style, meant she had been waiting.
"Eight of nine. What's the last one?"
"Supplier governance. Harcastle want to know whether we've assessed OpenAI, Microsoft, LexisNexis, and Experian as AI vendors. Whether we have contractual protections. Whether we've done due diligence on any of them."
A longer pause this time.
"Marcus. We use LexisNexis for AML. We've used them for years. We've never questioned what their AI does with our client data."
"I know."
"And Experian?"
"Same."
The last gap was not about what Whitmore built. It was about what they had signed — or, more precisely, what they had never thought to ask for.
ℹ Note — The HITL Gap Across Financial Services
Human-in-the-loop oversight failures are among the most common findings in FCA AI-related supervisory reviews. The most frequent pattern: firms have a general expectation that "someone reviews" AI outputs, but no documented criteria for what that review consists of, no evidence that it occurred, and no escalation path when the AI output is questionable. PROC-AIMS-FIN-HITL-001 addresses all three — but only if it is embedded into actual workflow, not filed and forgotten.
Gate 7: Story continuity from F-G7 (HITL placeholder fulfilled, day 14–16)
✓
Gate 8: No DANGER notices used
✓
Gate 9: Diana's voice used sparingly (one exchange, story-critical)
✓
Gate 10: Cliffhanger leads to F-G9 (vendor gap — never questioned LexisNexis or Experian)
Next in the Series — The Final Gap
Whitmore had been using LexisNexis for AML screening for years. They had never asked what the AI did with their client data. Never reviewed the contract. Never conducted due diligence on the model.
Five days. One question. And a supplier relationship that had never been treated as an AI governance obligation.
→ F-G9: AI Supplier Governance — The Final Question