Most teams meet governance as the last gate: build the pilot, then ask risk to sign. In a regulated organization that order is why pilots stall. This chapter puts governance first, because it is what lets the team move. Counts and durations on this page are illustrative.
Governance is not the brake on AI (artificial intelligence) adoption. It is the road. A team that knows its use case, its risk tier, its controls and its evidence can move every week. A team that does not know those things learns nothing until the final review says no.
1. Why governance is the first enabler
The organization already runs change control, separation of duties, model risk management, a privacy office, records management, third-party risk and internal audit. AI does not replace any of them. It changes the actor, the scale and the failure modes, so each existing control needs an extension, not a new bureaucracy.
| Without governance first | With governance first |
|---|---|
| Tool choice: each team picks its own assistant; nobody knows the data terms | One list of approved tools and configurations, with data terms checked once |
| Risk sign-off: arrives at the end and finds a prohibited data flow | Risk tier set at intake; the team knows the controls before it builds |
| Evidence: rebuilt by hand for every review | Produced by the work itself: intake card, evaluation report, pull request, change record |
| Second line: a reviewer of finished work | A partner who set the standard before work began |
| Speed: fast pilot, then months of rework | Slower first week, then a steady pace |
The point is not more approvals. It is that each approval happens at the right decision point, against a known standard, and is repeated when a material-change trigger fires.
2. Three lines of defense
The three lines of defense model is how the organization already divides risk work. AI adoption keeps the same division. What changes is what each line looks at.
| Line | Who | Role in AI adoption | Evidence it owns |
|---|---|---|---|
| 1st line | Delivery teams: business owner, product owner, developer, QA (quality assurance) engineer, release manager | Owns the use case and its risk. Files the intake, builds inside the guardrails, runs the evaluation, operates and monitors | Intake card, reuse decision record, evaluation results, pull request, change record, monitoring |
| 2nd line | Risk, compliance, security, privacy, AI / model risk | Sets the standard and the risk tiers. Challenges the 1st line. Validates independently where the tier requires it | Risk tier decision, control standards, threat model review, validation report, exception approvals |
| 3rd line | Internal audit | Tests that the controls work as designed and as operated. Independent of both other lines | Audit plan covering the AI inventory, sampled evidence chains, findings and follow-up |
Two rules keep the lines honest. The 1st line owns the risk; the 2nd line does not take it over by approving everything. The 3rd line never designs the control it will later test.
3. Two axes: maturity and risk
Every use case sits on two axes, and they are independent. Adoption maturity is how much a team can safely delegate to AI (the ladder, Stages 0 to 5). Use-case risk tier is how much harm a particular use case could cause, decided at intake.
| Stage | Name | AI does |
|---|---|---|
| 0 | Foundations | Nothing yet; inventory, intake, approved tools, policy, owners |
| 1 | Personal productivity | Explains, drafts, summarizes; the human applies every change |
| 2 | Team context | Grounded in the team's instructions file and shared prompts |
| 3 | Delegated tasks | Bounded tasks end to end; output is always a reviewable artifact |
| 4 | Agentic workflows | Multi-step workflows with tools, behind approval gates |
| 5 | Scale and sustain | Shared platform, catalog, continuous evaluation |
The risk tier is set per use case, not per team:
| Tier | Typical example | Needs |
|---|---|---|
| Low | Summarize public docs; explain code to its author | Approved tool, acceptable use |
| Medium | Coding assistant on proprietary source; internal policy assistant; internal decision support that a trained human independently verifies before acting | + data classification check, evaluation, traces, reviewer |
| High | AI output directly reaches a customer, or determines or materially drives an outcome for a customer (money, credit, regulated disclosures) | + privacy and AI/model risk review, independent validation, monitoring, kill switch, periodic review |
| Prohibited | Fully automated adverse customer decisions; sending restricted data to unapproved tools | Not allowed at any stage |
Internal decision support stays Medium only while a trained human independently verifies each consequential output before acting on it. It escalates to High if its output reaches customers directly or is used without that verification.
| Rule | What it means |
|---|---|
| No entitlement | A higher stage is not a licence for more autonomy. A high-tier use case may stay human-operated at Stage 5 |
| Prohibited stays prohibited | No stage, no team and no exception unlocks a prohibited use case |
| Tier travels | The tier belongs to the use case. Moving it to another team does not change it; changing its data or actions does |
| Step down | A model or provider change, a failed evaluation, an incident, a control gap or a material use-case change moves the team down until the evidence is back |
In the due-date example, the developer's coding assistant on proprietary source is a medium tier use case. The software change it helps write (how the due date shows on statements) is customer-facing, but it goes through the organization's normal change control, as any change would. The tier describes the AI use; existing controls still cover the change.
4. The control crosswalk
The crosswalk is the core of this chapter. For each control domain it reads left to right: the existing control the organization already runs, the AI failure mode it does not yet cover, the control extension, the evidence that proves it works, and the accountable owner. A blank, copy-ready row is on the templates page.
4a. Inventory, vendors and data
| Domain | Existing control | AI failure mode | Control extension | Evidence | Accountable owner |
|---|---|---|---|---|---|
| AI inventory and ownership | Application inventory and system owner register | Shadow use: an AI tool or use case runs with no owner, no tier and no record | Every AI use case is registered at intake: business owner, technical owner, tier, data classes, tools, status | Inventory entry linked to the intake card; yearly owner attestation | Business owner; the AI governance function keeps the inventory |
| Third-party and vendor risk | Vendor due diligence and contract review | The provider trains on the organization's data, adds a sub-processor, swaps the model without notice, or leaves no way out | Contract terms: no training on organization data, retention limits, sub-processor list with notice, provider change notice, audit rights, an exit plan with a tested fallback | Signed terms, independent assurance report, sub-processor register, exit plan test | Procurement and third-party risk |
| Privacy | Privacy impact assessment and purpose limitation | Personal data goes into prompts beyond the stated purpose; the context holds more than the task needs | The PIA (privacy impact assessment) covers the AI use; purpose is stated per use case; data is minimized and masked before it reaches the prompt | Completed PIA, masking test results, data-flow diagram | Privacy office |
| Data residency and cross-border | Data location rules and transfer assessments | Inference or logs run in an unapproved region; provider failover quietly moves data | Approved regions pinned in configuration; failover regions approved too; transfer assessment where applicable regulation requires one | Configuration record, region statement from the provider | Privacy and legal, with security |
| IP (intellectual property) and licence | Open-source licence policy and copyright review | Generated code reproduces licensed code; output ownership is unclear; confidential input reaches the provider | Licence and similarity scanning plus documented source and attestation where available; no match does not prove clean provenance; contract terms on output rights; rules on what confidential input is allowed | Licence and similarity scan report in CI (continuous integration); source or attestation note on the pull request | Legal, with engineering |
| Records and legal hold | Records schedule and legal hold process | Prompts and outputs that are business records are deleted too early, kept forever, or missed by a hold | Decide which prompts, outputs and traces are records; apply the schedule; make them reachable by a legal hold | Records classification, retention configuration, a hold test | Records management and legal |
Whether a prompt or an output is a business record is a records decision, not an engineering one; the team asks before it chooses a retention setting.
4b. Security and threats
| Domain | Existing control | AI failure mode | Control extension | Evidence | Accountable owner |
|---|---|---|---|---|---|
| Secure architecture | Identity and access management, network zones, secrets management, environment separation | The agent runs with a person's broad rights; secrets land in a prompt; a tool reaches production from a development session | Workload accounts for agents; least privilege per tool; secrets never in context; network egress allow-list; separate environments; a written tool permission list | Access review, tool permission manifest, secrets-scan results | Security |
| Threat model | Application threat modeling and penetration testing | Direct and indirect prompt injection, data exfiltration through output or tools, a poisoned knowledge source, excessive agency, a compromised model or plug-in supply chain | An AI threat model for every medium and high use case; red teaming; input and output filters; a source allow-list; human approval for consequential actions; pinned and verified dependencies | Threat model, red-team findings with fixes | Security |
The threat model deserves its own view, because AI adds threats that ordinary application reviews miss.
| Threat | What it looks like | First control |
|---|---|---|
| Direct prompt injection | A user types instructions that override the system prompt | Treat user text as data; narrow tools; output checks |
| Indirect prompt injection | A document, web page or ticket the agent reads contains hidden instructions | Source allow-list; no tool actions triggered by retrieved text alone; approval gates |
| Data exfiltration | The agent puts restricted data into an output, a link or a tool call | Data classification check; egress allow-list; output filters |
| Poisoned knowledge source | Someone edits an approved policy document, and the assistant now cites the wrong rule | Owned, versioned sources; change review on the corpus; citation checks |
| Excessive agency | The agent has more tools, rights or autonomy than the task needs | Least privilege; action boundaries; independent approval for consequential actions |
| Supply chain | A model, plug-in or tool server is replaced or compromised | Approved list; pinned versions; provider change notice; SBOM (software bill of materials) |
4c. Validation, operation and lifecycle
| Domain | Existing control | AI failure mode | Control extension | Evidence | Accountable owner |
|---|---|---|---|---|---|
| Validation proportional to risk | Model validation, testing standards, accessibility testing | Output is wrong, unstable, biased or unexplainable; validation is skipped because "it is only a tool" | An evaluation set with thresholds sized to the tier: quality, robustness, bias, explainability, accessibility; independent validation for the high tier | Evaluation report by risk slice; validator sign-off | AI / model risk, with QA |
| Monitoring, incident, kill switch | Production monitoring, incident management, business continuity | Quality drifts silently; nobody can stop the agent; a provider outage stops the process | Quality and drift monitors; a tested kill switch; rollback or a manual fallback; a provider outage runbook | Monitor dashboards, kill-switch test record, incident tickets | Business owner, with operations |
| Material change | Change control and model change policy | The provider swaps the model version, a prompt or knowledge source changes, or the use case widens, all without review | Material-change triggers are written down; the change re-enters triage; the evaluation runs again; the team may step down | Change record with the new evaluation result | Technical owner; AI / model risk confirms |
| Periodic review and retirement | Periodic model review and application decommissioning | The use case outlives its purpose; access, data and prompts linger | Review cadence by tier (for example yearly for medium and twice a year for high, illustrative); a retirement plan that revokes access, archives records and removes catalog entries | Review record, decommissioning checklist | Business owner |
| Evidence | Audit trail standards and logging policy | Logs exist but prove nothing: incomplete, editable, full of personal data, or impossible to find | Decide what is logged; protect integrity; restrict access; minimize content; retain by the records schedule | An evidence index linking intake, approvals, release and monitoring | Each control owner; internal audit tests it |
The lifecycle mechanics behind these rows (who runs intake, exceptions, incidents and retirement) are in the operating model.
5. Evidence that proves something
Logging is not evidence. A log is evidence only when someone can show it is complete, unchanged, readable by the right people, and kept for the right time.
| Property | Question it answers | Typical control |
|---|---|---|
| What is logged | Which events prove the control ran? | Intake decision, tier, approvals, evaluation run and result, hook and CI check results, model and prompt version, release, kill-switch use |
| Integrity | Can anyone change it after the fact? | Append-only store; write access separate from the team being evidenced |
| Access | Who can read it? | Role-based access; audit and 2nd line can read; the team cannot edit |
| Minimization | Does the evidence copy personal data it does not need? | Log references and hashes, not full prompts, unless the records decision says the prompt itself is a record |
| Retention | How long, and can a legal hold reach it? | The records schedule, not a default setting in a tool |
In the due-date example, the evidence chain runs: obligation, authoritative source, temporal decision, acceptance criteria, reuse decision, change, tests, approvals, change record, monitoring. Each link is an artifact the team already produces. The worked example walks it end to end.
6. Separation of duties
Separation of duties holds with agents in the loop. The agent changes who drafts the work; it never changes who approves it.
| Rule | What it means in practice |
|---|---|
| Requester is never approver | The person who asked the agent for a change cannot approve that change. The agent's output counts as the requester's work |
| Output lands as an artifact | Every AI step that changes a system lands as a reviewable artifact: a pull request, a ticket, a document. Nothing changes a system without one |
| Approver has the authority already | The approver is the person who holds that decision today, under the existing decision rights |
| Agent cannot self-approve | No agent identity holds approval rights. Branch protection and required reviewers enforce this, independently of any agent |
| Enforcement is not local | Agent hooks on a laptop help, but they can be bypassed. The pipeline and the repository rules are where enforcement lives |
7. Human approval is not automatically safe
A human signature does not make an AI output safe. Automation bias is the tendency to accept a machine's output because it looks confident and is usually right. A reviewer facing fifty agent pull requests a day (illustrative) will start to skim.
| The reviewer needs | Why | How to check |
|---|---|---|
| Time | A rushed review is a rubber stamp | Reviewer load per week; time spent per review |
| Competence | The reviewer must be able to spot the error the AI is likely to make | Named reviewers per domain; training on known failure patterns |
| Authority | The reviewer must be free to reject without escalation pressure | A rejection path that is quick and normal |
| Evidence | The reviewer judges the evidence, not the prose | Evaluation results, test results and the decision record attached to the artifact |
| Signal | The team must know if reviewers are just agreeing | Override and rejection rates; seeded test errors in review samples |
A very low rejection rate is a question, not a success. It may mean the agent is good, or that nobody is reading.
8. Existing decision rights stay
The rule is short: existing decision rights stay unless the risk assessment adds approval. The product owner still accepts stories. The release manager still schedules. Security still signs the controls.
AI can add an approval. A high-tier use case may need AI / model risk validation that the same change without AI would not need. AI never removes an approval. It also never moves a decision to a person who did not hold it before.
9. What counts as a "model"
Whether a use case is a "model" under the organization's model risk policy depends on that policy and on applicable regulation. One organization treats every large language model use as a model. Another treats a coding assistant as a tool and a customer-facing assistant as a model. Both can be right for their context.
| Question | Who answers |
|---|---|
| Is this use case in scope of model risk management? | AI / model risk, using the organization's own taxonomy |
| Which validation standard applies? | AI / model risk, by tier |
| Does the answer change if the use case changes? | Yes; that is a material change and goes back to triage |
This site uses "AI / model risk" to name the function. The organization uses its own name and its own scope.
10. What this is not
These are patterns and guidance, not legal or regulatory advice. Scope, terms and required controls follow each organization's own policy, its regulators and its jurisdiction. Tailor every table with the functions that own it.