Most AI governance arrives as a policy nobody can act on. This one starts at the other end — the operating model that runs every day, the people accountable for it, and one register everything else hangs off. Govern AI like any other critical capability: proportionate, risk-based, automated where it can be, accountable always.
Each move is a section below. Select one to jump to it.
A model with no owner is an incident with a delay. Name the owner before the use case ships.
You cannot govern what you don't know — and what you know needs different guardrails depending on how it reaches a person. What's in each population, and what you apply to it, in one view. Lettered A–E, and the same five carry through accountability (02) and tooling.
The register is the one thing that must exist before anything else works. Three honest paths — start where you are, not where you want to be.
A hand-typed register is stale in a month. Wire the feeds first, the fields second.
People reach for shadow AI because the approved path is slower. Fix the path, not just the policy.
Seven factors in, one tier out. Meeting summaries and autonomous clinical decisions should never meet the same committee.
The tier determines the controls. Not just a colour on a dashboard.
| Tier | Example | Governance approach | Key requirements |
|---|---|---|---|
Tier 1 — Low |
Meeting summaries, basic Q&A, non-sensitive data | Register + baseline controls | Acceptable use, data classification, logging, DLP, training |
Tier 2 — Moderate |
Internal RAG over company data, document analysis | Security & privacy review + testing | Access controls, data protection, prompt/RAG security, vendor review |
Tier 3 — High |
Customer chatbot, coding agent, HR AI | Threat model + AI red team + monitoring + approval | Comprehensive security testing, continuous monitoring, human oversight |
Tier 4 — Critical |
Medical or financial decisions, autonomous agents | Executive & risk approval + continuous assurance | Formal risk acceptance, continuous testing, human-in-the-loop, incident response |
If asking permission is harder than going around it, people go around it.
Agents can do things — so they need guardrails, not just guidance.
Your contract is with one company. Your data reaches four. Map the chain, put the answers in writing, take delivery of an AI-BOM, then keep watching — because the vendor's model will change without asking you.
Governance that lives in a document gets skipped. Governance that lives in the pipeline gets run.
A model that passed review in March is a different model in September. Watch it in production, and be able to reconstruct what happened.
Detection is not response. This part gets written before the afternoon an agent has already emailed four hundred customers the wrong price — or it does not get written at all.
| What went wrong | What it looks like | First containment move | Owns the call |
|---|---|---|---|
| Harmful or wrong output at scale | Fabricated fact, bad advice or offensive content reaching real users | Disable the route at the gateway and serve the fallback | Product owner |
| Sensitive data exposure | Regulated data in prompts, retrieved context, output or vendor logs | Quarantine the retrieval index; suspend the connector | AI Security + Privacy |
| Prompt injection or tool abuse | Untrusted content steering the model into calls nobody intended | Quarantine the content source; drop the tool from the manifest | AI Security |
| Agent overreach | Actions beyond intent — spend, send, delete, commit, escalate | Revoke the agent identity; freeze its entitlements | AI Security + Engineering |
| Upstream change or compromise | Provider silently changed the model, poisoned dependency, vendor incident | Pin to the last known-good version; fail over to the fallback model | Engineering + vendor owner |
| Runaway loop or cost | Recursion, retry storms, denial of wallet | Enforce the spend cap; kill the session | Engineering |
| Severity | Trigger | Acknowledge | In the room | External clock |
|---|---|---|---|---|
S1 — Critical |
Real-world harm, regulated data confirmed out, or an agent committed a financial or safety-relevant action | 15 min, 24×7 | Incident commander, exec sponsor, Legal, Privacy, Comms | Starts now |
S2 — High |
Exposure suspected but bounded, or wrong output to a limited, identifiable population | 1 hour, on-call | AI Security, product owner, engineering lead | Legal on standby |
S3 — Moderate |
Policy breached, no confirmed exposure or harm | Next business day | AI Security, product owner | Internal only |
S4 — Near miss |
Caught by an eval, a guardrail or a user before it reached anyone | Backlog | Owning team | None — feeds Assure |
There is no stack trace. The failure lives in a prompt, a retrieved document and a model version, and the same input may not reproduce it tomorrow.
The fix is often a prompt change — which means the fix is a change nobody versioned, unless prompts were already release artefacts.
The vendor can change the model underneath you between the incident and the review, and take your ability to reproduce it with them.
The evidence is gone unless it was logged at the time. Nothing here is recoverable after the fact, which makes 4.1 the precondition for this whole page.
Assurance is a subscription, not a purchase. Prove it before it ships, then keep proving it.
Four things move under you: the data, where it came from, the model, and where it came from. Two need a signal. Two need a signed record.
Did the input distribution move?
Where did it come from, who touched it, what did it feed?
Is it still the model that passed review?
Is this model what it claims to be?
Build it once, evidence it many times.
| Practice | NIST AI RMF | ISO/IEC 42001 | EU AI Act |
|---|---|---|---|
| 01 Operating model | GOVERN 1 | Cl. 4–10 · A.2 · A.6 | Art. 17 |
| 02 Accountability | GOVERN 2, 3 | A.3 | Art. 4 · 26 |
| 03 Inventory & lane controls | MAP 1, 4 · MANAGE 2 | A.4 · A.6 · A.7 | Art. 10 · 11 · 15 · 49 |
| 04 Risk engine | MAP 5 · MEASURE 1 | A.5 | Art. 6 · Annex III |
| 05 Risk tiers | MANAGE 1 | A.5 · A.9 | Art. 9 |
| 06 Agentic control plane | MANAGE 2.3 · MEASURE 2 | A.6.2 · A.9.3 | Art. 14 · 15 |
| 07 Vendor AI | GOVERN 6 · MANAGE 3 | A.10 | Art. 25 · 26 |
| 08 Shadow AI | GOVERN 1.6 · MAP 4.1 | A.4.2 · A.9.2 | Art. 4 · 26 |
| 09 Engineering lifecycle | MEASURE 2 · MANAGE 4 | A.6 | Art. 15 · 17 |
| 10 Continuous assurance | MEASURE 3, 4 | A.6.2 | Art. 72 · 73 |
| 11 Approval workflow | GOVERN 1.3 · MANAGE 1.1 | A.5.4 · A.9.2 | Art. 9 · 26 |
An assessor starts from their framework, not yours. MAP, MEASURE and MANAGE run the lifecycle; GOVERN sits across all three, which is why it comes last here and first in a maturity conversation.
| Function | Category | What it asks for | Delivered by |
|---|---|---|---|
| MAP — understand the system in context | |||
| MAP 1 | Context, purpose and users established | 03 Inventory | |
| MAP 2 | AI system categorized | 04 Risk engine · 05 Tiers | |
| MAP 3 | Capabilities, targets, costs and benefits | 11 Approval intake | |
| MAP 4 | Risks mapped across all components, third party included | 03 Inventory · 07 Vendor AI | |
| MAP 5 | Impacts to individuals, groups and society | 04 Decision consequence factor | |
| MEASURE — test it, and keep testing it | |||
| MEASURE 1 | Methods and metrics identified | 09 Lifecycle tests · 10 Assurance | |
| MEASURE 2 | Evaluated for trustworthy characteristics | 09 Build / test · 10 Before production | |
| MEASURE 3 | Mechanisms for tracking identified risks | 10 Runtime monitoring | |
| MEASURE 4 | Feedback on whether the measurement works | 01 Assure | |
| MANAGE — act on what you found | |||
| MANAGE 1 | Risks prioritized and responded to | 05 Tiers · 11 Approval | |
| MANAGE 2 | Strategies to maximize benefit, minimize harm | 03 Lane controls · 06 Agentic control plane | |
| MANAGE 3 | Third-party risks managed | 07 Vendor AI | |
| MANAGE 4 | Treatments documented and monitored, incidents handled | 10 Detect & respond · 01 Monitor | |
| GOVERN — cross-cutting, sits over all three | |||
| GOVERN 1 | Policies, processes and procedures in place | 01 Operating model · 11 Approval | |
| GOVERN 2 | Accountability structures and named roles | 02 Accountability | |
| GOVERN 3 | Workforce competency and team composition | 02 Accountability — thin | |
| GOVERN 4 | Culture of risk awareness | 08 Shadow AI coaching | |
| GOVERN 5 | Stakeholder engagement and feedback | 10 User feedback — thin | |
| GOVERN 6 | Third-party and supply chain policy | 07 Vendor AI | |
They are not competing options. Buy the must-haves, borrow from the nice-to-haves, and know what none of them covers.
Every red team tool in this deck — PyRIT, garak, promptfoo — tests a prompt. Agents fail somewhere else: on the third tool call, in a delegation chain, when state from turn one poisons turn nine. Nothing in ISO 42001, the AI Act, NIST AI RMF or the SSDF tells you how to test that, what coverage looks like, or when to stop. Even AICM and the OWASP agentic list describe the risk without prescribing the test.
Which means you write the method yourself. Define your own trajectory test suite, your own tool-misuse cases, your own pass criteria — and document that you invented them, because an assessor will ask which standard you followed and there isn't one. Say so plainly rather than mapping it to a control that doesn't fit.
Each move in the operating model, turned into something a named person does on a trigger, produces a record for, and can be held to. One accountable owner per row — never two.
| Move | Trigger | Accountable | Produces | Target | If it doesn't happen |
|---|---|---|---|---|---|
| Discover | Continuous feeds, or an intake form | AI Security → product owner on registration | Register entry with a named owner | Before first use | You govern a fraction of what exists, and nothing has an owner |
| Decide | Registration complete | GRC tiers · AI Security approves T1–2 · Council approves T3–4 | Tier with rationale, decision, conditions | 1 / 5 / 15 days / monthly | Critical systems get trivial scrutiny, and people route around you |
| Control | Approval with conditions | Engineering | Controls live, gated in the pipeline | Before go-live | Approved on paper, unprotected in production |
| Monitor | Go-live, then alerts | SOC, escalating to the product owner | Telemetry in the SIEM, incident records | Continuous · triage 1h · owner 4h | A customer tells you first, and the clock has already run |
| Assure | Schedule, or any material change | AI Security, with GRC on evidence | Test results, attestation, updated tier | Quarterly for Tier 3–4 | Your evidence describes a system that no longer exists |
One A per row. If two people are accountable, nobody is.
| Activity | Product owner | Eng | AI Security | Privacy | Legal | Procure ment | GRC | SOC | Exec council |
|---|---|---|---|---|---|---|---|---|---|
| Discover AI in use | I | C | A | I | — | C | I | R | I |
| Register a system | A | R | C | C | — | — | R | — | I |
| Assign the risk tier | C | — | C | C | C | — | A | — | I |
| Approve Tier 1–2 | R | I | A | C | — | — | I | — | — |
| Approve Tier 3–4 | R | C | R | C | C | C | R | — | A |
| Threat model & red team | C | R | A | — | — | — | I | C | I |
| Apply lane controls | C | A | R | C | — | — | I | C | — |
| Agent authorization design | C | R | A | — | — | — | I | C | I |
| Vendor assessment & AI-BOM | C | C | R | R | R | A | C | — | I |
| Shadow AI decision | C | — | A | C | — | C | I | R | — |
| Pipeline gates | I | A | R | — | — | — | C | — | — |
| Runtime monitoring | I | R | R | — | — | — | I | A | — |
| AI incident response | R | R | R | C | C | — | C | A | I |
| Re-review & re-tier | R | C | C | C | — | C | A | — | I |
| Report to the board | I | — | C | C | C | — | R | — | A |
Governance that only happens at approval is a gate. Governance on a calendar is an operating model.
Twelve failure modes that show up in real programmes. Each has a lane, a control that addresses it, and one accountable owner.
| Risk | Lane | What stops it | Accountable |
|---|---|---|---|
| Sensitive data pasted into an unsanctioned tool | E | Discovery, DLP, approved alternatives | AI Security — lane E |
| Prompt injection leads to data exfiltration | B | Input validation, output controls, gateway | Product Security |
| Agent takes an action nobody authorized | C | Policy enforcement point, least privilege | AI Security — lane C |
| Vendor changes the model without telling you | D | Version pinning, canary evals, notice clause | Procurement |
| Your data trains someone else's model | D | Contract terms, DPA flow-down, AI-BOM | Legal |
| RAG returns records the user shouldn't see | B | Entitlement-aware retrieval, index scoping | Engineering |
| Output drifts and a real decision goes wrong | B | Golden-set evals, drift monitoring, human review | Product owner |
| AI reaches production unregistered | A | Pipeline gate on register ID | Engineering |
| A high-risk system ships with no impact assessment | A | Tiering at intake, approval gate | GRC |
| No provenance for a model or dataset | D | AI-BOM at every release, signing | Product Security |
| Agent credentials escalate into other systems | C | Agent identity, scoped tokens, kill switch | IAM |
| An incident happens and there are no logs | A | Observability as a go-live condition | SOC |
The same operating model, read from five different desks.
Order matters more than completeness. Each window assumes the one before it.