The review that happened but cannot be proved
Every fee earner at Halstead & Cole LLP who uses GPT-4o for case chronologies will tell you they review the output before using it in a matter. They are not lying. They do read it. But "reading it" is not a review standard, and a review with no standard produces no audit trail, no evidence of what was checked, and no mechanism for catching the specific failure modes that AI systems are actually prone to — fabricated citations, transposed dates, subtly wrong legal propositions that read correctly at speed.
Ayinde v London Borough of Haringey [2025] EWHC 1383 (Admin) is instructive precisely because the submissions were presumably read before filing. The problem was not an absence of eyes on the document. It was an absence of a structured review that would have caught the specific error type — hallucinated case citations — that large language models produce with documented regularity in legal contexts. A structured checklist that includes "verify every case citation against the primary source" would have caught it. An unstructured read-through did not.
SRA Principle 5 requires proper supervision of legal work. The SRA's guidance of 9 February 2026 makes clear that this obligation extends to AI-generated content. Supervision has a specific meaning in regulatory terms: it is not simply the presence of a senior person who could theoretically have intervened. It is a process — a defined standard of review, applied consistently, with evidence that it occurred. An undocumented review that leaves no trace in the matter file is not supervision under SRA Principles. It is supervision in name only — and the distinction between the two carries conduct and enforcement consequences.
This article explains what Clause 8.4 requires by way of human oversight, why the three-tier oversight model is the right framework for law firm AI systems, and how PROC-AIMS-HITL-001 — the human-in-the-loop procedure — closes Gap 8 with a review standard that is proportionate, evidenced, and integrated with every prior gap document.
Clause 8.4 and the oversight obligation — with Clause 9.1 as the documentation anchor
The organisation shall implement appropriate human oversight mechanisms for AI systems, taking into account the AI system risk level and the context of use. Human oversight mechanisms shall be documented and implemented to ensure that humans can intervene in, correct, or override AI system outputs where necessary.
The organisation shall define the roles and responsibilities for human oversight and ensure that persons assigned oversight responsibilities have the competence to fulfil those responsibilities.
Clause 8.4 is the human oversight clause of ISO 42001. Its core obligation — documented oversight mechanisms that ensure humans can intervene, correct, or override AI outputs — is the requirement that undoes most law firms' informal review approach. A fee earner reading an AI output and deciding it is good enough is an oversight action. The absence of any record of that action means the Clause 8.4 standard has not been met.
Clause 9.1 (Monitoring, Measurement, Analysis and Evaluation) is the documentation anchor that amplifies Clause 8.4's obligation. Its requirement — retain appropriate documented information as evidence of the results — means the oversight must leave a trace in the matter file. Read together, Clauses 8.4 and 9.1 imply three specific obligations for law firm AI systems:
A defined review standard for each risk tier. The "what needs to be monitored" question shall be answered specifically — not just "AI output quality" but the specific error types the system is known to produce in this use context, and the specific checks required to detect them. This is the checklist in PROC-AIMS-HITL-001.
Documented evidence per matter. The "retain appropriate documented information" requirement means the review shall leave a trace in the matter file — a sign-off record showing who reviewed, what they checked, and what the outcome was. This is the HITL sign-off form in the procedure.
Aggregated performance reporting. The "analyse and evaluate results" requirement means COLP Priya Anand and the AGL shall receive periodic reports on AI output quality across the practice — not just per-matter sign-offs but a pattern view that identifies systemic issues. This feeds into the quarterly AIMS performance report required by RACI-AIMS-001.
Annex A.6 (human oversight) is a control set within ISO 42001's recommended controls. Clause 8.4 (human oversight of AI systems) is a core requirement. The distinction matters: Annex A.6 controls are recommended but not mandatory — firms can justify omitting them if they document why they are not applicable. Clause 8.4 is mandatory — the obligation to implement documented oversight mechanisms admits no exception. PROC-AIMS-HITL-001 satisfies both simultaneously: the three-tier review model implements Annex A.6; the sign-off records satisfy Clause 9.1's documentation requirement.
Primary mapping — Service Validation and Testing: ITIL 4's Service Validation and Testing practice ensures that services meet their defined requirements before and during use. PROC-AIMS-HITL-001's Tier 1 checklist is, in ITIL 4 terms, a service validation procedure applied at the point of output consumption — each use of a HIGH-risk AI system in a client matter is a service delivery event that shall pass validation before the output reaches the client. The 10-item checklist maps to ITIL 4's acceptance criteria framework: each item is a defined acceptance criterion that the output shall satisfy before it is used. A failed checklist item is a service validation failure — it triggers the incident escalation pathway, not a corrective note in the margin.
Secondary mapping — Continual Improvement: The five-step incident escalation pathway — from fee earner correction through COLP assessment, risk register update, and quarterly pattern detection — is the ITIL 4 Continual Improvement model applied to AI oversight. Each detected error is a data point in the measurement cycle (Step 1); Material errors trigger analysis (Steps 2–3); pattern detection triggers structural improvement (Steps 4–5). The quarterly AIMS performance report is the Continual Improvement review artefact that prevents individual error corrections from remaining invisible at the system level. Without it, the firm corrects the same error type repeatedly without ever improving the procedure that generates it.
The SRA's supervision standard applied to AI output
The SRA requires that AI use does not diminish the professional obligations of solicitors — AI-generated work product shall be supervised to the same standard as work produced by any other means.SRA · Compliance Tips for Solicitors Regarding the Use of AI and Technology · 9 February 2026
The SRA's position is unambiguous: AI output is work product, and work product requires supervision. The question is what supervision means in practice when the "drafter" is a language model rather than a trainee solicitor.
For a trainee solicitor, supervision means: a qualified solicitor reviews the work, checks it for accuracy and legal correctness, identifies errors, and records their sign-off. The standard for AI output is the same — with one addition. Because AI systems have documented, specific failure modes (hallucination, bias, scope drift, citation fabrication), the review shall include checks calibrated to those failure modes, not just to the general quality standards applied to human-drafted work.
A fee earner who "reads" an AI-generated chronology is doing the equivalent of a supervising partner glancing at a trainee's draft. That is not supervision — it is awareness. Supervision requires: knowing what to look for, systematically checking for it, and leaving a record that the check was done. PROC-AIMS-HITL-001 operationalises this standard for each of Halstead & Cole LLP's registered AI systems.
The three-tier oversight model and the four failure patterns
The oversight requirement is not uniform across all AI systems. A risk-proportionate model — derived from the risk register scores in REG-AIMS-RISK-001 — applies different review standards to High, Medium, and Low risk systems. This is the three-tier model that PROC-AIMS-HITL-001 implements.
The four failure patterns oversight must catch
The four oversight failure patterns in practice
PROC-AIMS-HITL-001: the human-in-the-loop procedure
The procedure has four operational components: the three-tier review standard, the Tier 1 review checklist, the matter-file HITL sign-off form, and the incident reporting and escalation pathway. Together they close all four non-conformances identified above.
The Tier 1 review checklist — GPT-4o and HIGH-risk systems
This is the core instrument — the structured checklist that turns an unstructured "read-through" into a documented, evidenced review. Every GPT-4o output used in a client matter shall be reviewed against this checklist before use.
The incident reporting pathway
When the Tier 1 checklist detects a material error — a fabricated citation, a significantly wrong date, a materially incorrect legal proposition — it is not simply corrected and forgotten. The incident escalation pathway in PROC-AIMS-HITL-001 routes the finding through the governance framework.
The distinction between Minor, Material, and Serious errors determines the escalation path — it does not determine whether the error is recorded. Every error detected during a Tier 1 review shall be recorded in the HITL Sign-Off Form, regardless of severity. Minor errors that recur become Material patterns. The pattern is only visible if the individual instances are recorded. A firm that only records serious errors will systematically miss the early-warning signal that precedes them.
PROC-AIMS-HITL-001: Human-in-the-Loop Oversight Procedure
The controlled document that closes Gap 8 includes: the three-tier oversight procedure with role assignments and cadence for each tier, the Tier 1 review checklist (printable, matter-file ready), the HITL Sign-Off Form (per-output, captures reviewer, checklist outcome, errors detected, and escalation decision), the Tier 2 spot-check log, the incident escalation procedure with severity classification, and the quarterly performance reporting template for COLP Priya Anand's AIMS report.
Three-tier oversight model with role assignments · Tier 1 review checklist (10 checks, printable) · HITL Sign-Off Form per output · Tier 2 spot-check log · Incident escalation procedure with F1–F4 error classification · Quarterly performance reporting template · Version control and COLP sign-off.
Implementation priority: Brief all fee earners on the Tier 1 checklist before the next time GPT-4o is used in any client matter. The checklist is the most immediately consequential instrument — it is what converts the risk register's "human review of every output" control from an aspiration into an evidenced practice. The HITL Sign-Off Form takes 2–3 minutes per output; the discipline of completing it is more important than its content in the first month of implementation.
G8 completes the operational controls — one gap remains
With G8 in place, the AIMS operational architecture is complete for Halstead & Cole LLP's internal AI systems. Every system in the register now has: a risk assessment (G4), a lifecycle status (G5), data governance controls (G6), a client disclosure framework (G7), and a documented human oversight procedure (G8). The remaining gap — G9, supply chain governance — addresses the second scenario firm, Thornbury Legal Ltd., whose self-hosted AI model creates obligations that the preceding gaps do not fully cover.
The final article in the gap series covers G9: supply chain and third-party AI governance under ISO 42001 Clause 8.5. For firms that procure AI capabilities through multiple vendors, aggregators, or self-hosted deployments, the supply chain is where shadow risk concentrates — and where most governance frameworks end before they should.
Quality gate record
The following gates were checked before publication. All 10 gates pass. This record satisfies ISO 9001:2015 Clause 7.5.3 (control of documented information) for the article series. Template: TMPL-AIMS-ART-001 v1.0.
- G1All regulatory claims carry named, citable sources. SRA instrument cited as Compliance Tips for Solicitors Regarding the Use of AI and Technology, 9 February 2026. ISO/IEC 42001:2023 Clauses 8.4 and 9.1 cited directly. Ayinde v London Borough of Haringey [2025] EWHC 1383 (Admin) cited in full in stat bar and body. Stat bar figures attributed to PROC-AIMS-HITL-001 v1.0 and UNUS London baseline assessments 2026. 5× statistic removed — no verifiable source available; replaced with attributable 10-point checklist count.PASS
- G2Primary gap clause corrected throughout from Cl. 9.1 to Cl. 8.4 (Human Oversight of AI Systems), consistent with TMPL-AIMS-ART-001 Part E registry. Cl. 9.1 retained as the documentation anchor in Section 2 body text, correctly characterised as the subordinate measurement and evidence obligation. British English used in clause callout ("organisation" not "organization"). Prescriptive language — "shall" for requirements — applied correctly throughout.PASS
- G3Scenario internally consistent with REF-AIMS-SCEN-001 v1.0. Firm: Halstead & Cole LLP. COLP: Priya Anand (Partner). AI systems: AIMS-SYS-001–004 correctly named with risk tiers. Second scenario firm: Thornbury Legal Ltd. No new named characters or firms introduced.PASS
- G4ISO/IEC Directives Part 2 prescriptive language applied correctly. SRA supervision obligation characterised as "requires" throughout. Pull quote replaced with paraphrase and attribution. Hero deck updated from "expects" to "requires." Procedure obligations use "shall" consistently.PASS
- G5PROC-AIMS-HITL-001 download linked ungated in Section 6 doc-cta block. All cross-referenced document IDs present and correct: POL-AIMS-001, REG-AIMS-SYS-001, REG-AIMS-RISK-001, PROC-AIMS-LIFE-001, REG-AIMS-DATA-001, DOC-AIMS-DISC-001, RACI-AIMS-001, REG-AIMS-SUP-001.PASS
- G6Footer includes predecessor link (← G7 — Client Disclosure) and successor link (G9 — Supply Chain →). Section 7 documents the downstream dependency (G9) with document ID and ISO clause reference. Breadcrumb updated to /legal/ path structure.PASS
- G7Standalone disclaimer section present between final-cta and footer. Includes: educational purpose notice, fictional scenario notice for Halstead & Cole LLP, Priya Anand, and Thornbury Legal Ltd., document reference ART-AIMS-G008 v1.1, publication date, next review date, retention period, and canonical URL.PASS
- G8No inline policy or template content requiring a document-control block. PROC-AIMS-HITL-001 is described and linked ungated. The Tier 1 checklist preview is an editorial illustration, not a reproduction of the controlled document. Gate marked N/A.PASS
- G9H1 includes primary keyword "human oversight of AI" within the title. Stat bar appears within first screenful. Schema.org Article JSON-LD with isPartOf series reference present in <head>. Canonical URL set: https://unuslondon.com/legal/iso-42001-gap-8-human-oversight. Meta keywords present (6 terms including Cl. 8.4 reference and "law firm").PASS
- G10IEC 82079-1:2012 notice hierarchy CSS comment block present above :root. .notice-danger class absent from stylesheet — removed entirely. DANGER not used anywhere. Section 1 notice reclassified from DANGER to WARNING (SRA supervision enforcement consequence). Incident escalation flow reclassified from danger palette to CAUTION palette (procedural risk); danger symbol "⚑" replaced with CAUTION symbol "◈" in label. CAUTION used for "Minor errors recorded" risk (Section 5). NOTE used for Cl. 8.4 vs Annex A.6 distinction (Section 2). ITIL 4 NOTE present with indigo styling, mapped to Service Validation and Testing and Continual Improvement (Section 2). Tier 1 colour corrected from red danger palette to warning amber. Tier 3 corrected from green (CAUTION) to blue (NOTE/informational) — periodic audit is informational, not a risk state.PASS