Treat model, prompt, corpus and tool changes as changes; verify the evidence chain before any release.
The release manager decides whether a change is ready to reach production, and on what evidence. AI (artificial intelligence) adds new kinds of change: a model version, a prompt, a retrieval corpus, a tool, a policy rule, an evaluation threshold, a provider. Each of these is a change, even when no line of application code moves.
This track shows how the release manager adopts AI in their own work, and how they release systems that contain AI. Counts and durations on this page are illustrative.
1. At a glance
Item
Answer
Mandate
Only evidenced, approved, reversible changes reach production. AI components are changes like any other
Primary decision
Authorize the change, authorize it with conditions, or hold it, under existing change control
First safe win
AI drafts the change record and release notes from linked evidence; the release manager checks every claim against its link
Biggest failure mode
A prompt, model or corpus change ships outside change control because "no code changed", or generated release notes stand in for evidence
Line: 1st line · Family: Release and run · Variants: release manager, change manager, change advisory chair
Role variants
Variant
What differs
Release manager
Plans and coordinates the release: sequencing, flags, rollout phases, rollback readiness, support readiness
Change manager
Owns the change process itself: categories, the material-change assessment, the change calendar, change authority by category
Change advisory chair
Runs the review for changes that need a group decision; records who decided and why. Does not replace the approvers each control names
2. Your journey: first steps to fully adopted
The release manager moves from making AI changes visible to checking evidence at scale; the go or hold decision stays with a person at every stage.
Start here: your first steps
Add model, prompt, corpus, tool, policy rule, threshold and provider to the change categories, so no AI change ships as "no code changed".
Copy the AI change readiness check from section 14 into the change record template.
Use an approved tool to summarize a long PR (pull request) thread or draft a status note, with no customer data in the prompt.
Write a reusable, reviewed prompt that drafts release notes and refuses any claim without an evidence link.
Fully adopted: how the work runs
Moment in the work
Before AI
Fully adopted (Stage 4–5)
Decision that stays human
Change record
The release manager writes it by hand and chases missing links
An agent assembles a draft record from linked PRs, tests and evaluation runs, and lists every gap
Whether the record is complete enough to submit
Readiness
A meeting reads a checklist aloud
A shared evidence checker confirms each link resolves, the evaluation run matches this exact configuration, and the rollback rehearsal is on record
Whether the evidence is sufficient; the material-change routing
Rollout
One big release to everyone
A workflow schedules phases by flag and posts post-release check results after each phase
Opening the next phase
Go or no-go
Decided on fluent release notes and confidence in the room
Decided on a complete evidence chain; the release notes only link to it
Go, conditional go or hold
Rollback
A rollback plan exists but nobody has rehearsed it
Rollback or kill switch rehearsed before go; a failed post-release check surfaces the flag-off step from the change record
Calling the rollback, with production support
What changes for the organization
Before
After
A prompt edit or model swap ships outside change control
Every AI component change has a change record and its own evaluation run
Missing evidence surfaces late, at the review
Gaps surface when the draft record is built, while there is time to fix them
Release notes are trusted as proof
Every claim links to evidence that audit can sample later
In any organization
Small or unregulated: a tech lead or engineering manager holds the release role; a second person who did not write the change approves it.
What scales down: the change board, formal categories and validation shrink to a checklist. Treating AI components as changes, an approver who is not the author, and a tested rollback never do.
3. Your decisions
Decision
You own, advise or challenge
Inputs
Escalation
Cannot be delegated
Is this a change, and which category?
Own
Change description, the list of AI components touched, the material-change triggers
Change manager for a disputed category
Yes
Is it a material change for the AI use case?
Own the routing; AI / model risk confirms the outcome
Material-change assessment, new evaluation run, tier from intake
AI governance function, back to triage
Yes
Is the evidence chain complete?
Own
Links to criteria, reuse decision, tests, evaluation, control reviews, validation where required, approvals
The accountable owner of the missing link
Yes
Rollout shape: phases, flags, cohorts
Own, with the technical owner
Risk tier, blast radius, monitoring readiness
Business owner for customer-facing timing
No: the technical owner may propose; you authorize
Rollback and kill-switch readiness
Own the check; engineering owns the mechanism
Kill-switch test record, rollback plan, fallback runbook
Production support lead
Yes
Go, conditional go or hold
Own, within your change authority
All of the above, plus support readiness
Business owner and change authority for high-tier changes
Yes
Release-note wording for customers
Advise; compliance approves the wording
Draft notes, the change record
Compliance
Yes for the approval; drafting may be AI-assisted
4. Two kinds of adoption
Kind
What it means for the release manager
Example
How you work with AI
AI drafts change records, release notes, rollout plans and risk summaries from linked evidence. It flags missing links. You verify and decide
AI lists the four pull requests for the due-date change and flags that one has no test-matrix link
What you release that contains AI
Systems where a model, prompt, corpus, tool, policy or threshold is part of the product. Every one of these is a versioned configuration item under change control
The internal policy assistant's policy index is refreshed; that refresh is a change with its own evaluation run
The second kind matters more. A team can move a model version, rewrite a system prompt, or add a tool without touching application code. If the change process only watches code, those changes ship unseen.
AI component
Why it is a change
Minimum evidence before release
Model or model version
Behavior changes even when the prompt does not
Full evaluation run on every risk slice; provider change notice; AI / model risk confirmation
Prompt or system prompt
Wording changes behavior, refusals and abstention
Diff in a PR (pull request); evaluation run; reviewer who did not write it
Retrieval corpus or index
New, removed or edited documents change answers and citations
Corpus diff with document owners; citation and stale-document tests
Tool or tool scope
A new tool or a wider permission changes what the system can do
Owner of go or hold; production support is consulted on readiness
Any action outside the tool list, or a missed rollback check
5 · Scale and sustain
Shared change-record prompt and evidence checker in the asset catalog; continuous checks across teams
Asset owner, versioning, periodic review of the checker
Catalog entry; checker evaluation trend
Catalog owner approves the asset; you remain owner of each release
A model or provider change for the checker itself
Stays human at Stage 5: the go, conditional-go or hold decision, and the material-change routing. An agent may check that evidence exists. It does not decide that the evidence is sufficient. A team steps down after a model or provider change, a failed evaluation, an incident, a control gap or a material change to the use case; see the adoption ladder.
6. Priority scenarios
Scenario
Value
Risk tier
Stage
First step
Do not use AI when…
Draft release notes from linked evidence
Hours saved per release (illustrative: 2 to 3)
Medium
2
Write a reusable prompt that refuses to state a claim without a link
The notes are the only evidence a reviewer will see
Evidence-completeness check
Fewer late surprises at the change review
Medium
3
Build an evaluation set of 30 past change records with seeded gaps (illustrative)
The checker's own pass is treated as approval
Material-change triage helper
Prompt, model and corpus changes stop slipping through
Medium
3
List the triggers in a checklist the helper must cite
A human has not confirmed the category
Rollout plan draft
Consistent phases and flags
Medium
3
Template with cohorts, flags and rollback per phase
The change is high tier and the rollout is not yet agreed with the business owner
Post-release check summary
Faster go-forward or rollback call
Medium
4
Read-only access to the monitoring signals named in the change record
A signal is missing; then the check fails, it is not summarized away
Customer-facing wording
Clear, consistent notices
High when the text reaches customers
2
Draft only; compliance approves
The wording is a regulated disclosure and no compliance review is booked
7. The due-date ask from your seat
The due-date change is a material artifact for this role, so it gets the full narrative. The change touches four display channels and an additive library extension. The AI use behind it (a coding agent that drafted code, tests and notes) stays medium tier, because no model is in the runtime path and no AI output reaches a customer.
Step
What the release manager does
Evidence checked
Category
Classifies a customer-facing normal change. Confirms payment-schedule-service and profile-service do not change
Impact analysis linked to the change
AI involvement
Records that a coding agent drafted code, tests and the note draft, with traces linked
A "no material change" claim with no new evaluation run
Exceptions that expire before the release date
Material-change routing → AI governance function
Threat model and tool permission review ← security
Release notes that describe behavior the tests did not cover
Audit requests for sampled change records
Separation of duties: the person or agent that prepared a change never approves it. An agent can draft the change record; it has no permission to submit, approve or deploy. Where one person holds two roles in a small team, a second approver from outside the team signs consequential changes.
9. Evidence and records
Evidence produced or consumed
System of record
Retention owner
Change record, with AI involvement noted
Change management system
Release manager, under the change records schedule
Design phased rollouts with flags and kill switches for AI features; set evidence-completeness checks as a shared asset
Lab L4 (evaluation sets) and Lab L6 (policy assistant corpus change)
Scenario practice matters more than prompting skill here. Run three tabletop drills (illustrative cadence: once a quarter): a silent provider model swap, a corpus refresh with a withdrawn document, and a threshold lowered the night before release.
11. Measures and incentives
Metric
Definition
Anti-gaming note
Untracked AI changes
AI component changes found in production with no change record, per quarter
Count what monitoring and audit find, not what teams self-report
Evidence completeness at review
Share of changes with every required link present when review opens
Links must resolve to real artifacts; a link to an empty page counts as missing
Change failure rate
Share of releases that cause an incident or rollback
Do not reward zero rollbacks; a fast, clean rollback is a control working
Rollback readiness
Share of AI changes with a tested rollback or kill switch before go
Tested means a recorded rehearsal, not a plan
Draft acceptance with edits
Share of AI-drafted records accepted after review, and how much was edited
High acceptance with no edits is a warning sign of automation bias, not a win
12. Cost, capacity and workforce
Item
What to plan
Capacity
AI drafting saves time on records; reinvest it in evidence checks, not in more releases per reviewer
Cost
Evaluation runs per release cost money and time; budget them per AI change, not per quarter
Workforce
The role shifts from writing records to verifying them. Train for line-by-line review and scenario judgment
13. Anti-patterns and stop conditions
Anti-pattern
Why it hurts
Stop condition
"No code changed, no change"
Prompt, model and corpus changes ship unseen
Any AI component change with no change record: stop the release, open one
Release notes as evidence
A fluent summary hides a missing test
A claim in the notes with no link: hold
Agent as approver
Breaks separation of duties
Any agent with approve or deploy permission: revoke before the next release
Threshold moved to pass
Lowers the bar without a decision
Threshold change in the same release as the behavior it gates: hold for the threshold owner
Big-bang rollout of an AI feature
No early signal, large blast radius
High-tier AI change with no phase plan: hold
Kill switch never tested
It fails when needed
No rehearsal record in the last review cycle: conditional go at most
14. Action kit and 30/60/90
AI CHANGE READINESS CHECK (copy into the change record)
[ ] AI components touched: model / prompt / corpus / tool / policy rule / threshold / provider
[ ] Material-change assessment done; outcome confirmed by AI / model risk if triggered
[ ] Evaluation run on this exact configuration; result on every risk slice; link
[ ] Control reviews linked: security, privacy, compliance as the tier requires
[ ] Independent validation conclusion linked (high tier)
[ ] Approver is not the author; no agent approval
[ ] Phase plan with flags and a monitor per phase
[ ] Rollback or kill switch rehearsed; record linked
[ ] Production support briefed; runbook updated
[ ] Release notes: every claim links to evidence; customer wording approved by compliance
Questions to ask your enablement lead:
Which AI components does our configuration item list already track, and which are missing?
Is there an approved, evaluated prompt for change-record drafting I can reuse from the catalog?
Who confirms a material-change outcome for each tier, and how fast?
Window
Goal
Done when
30 days
Make AI components visible as changes
Change categories name model, prompt, corpus, tool, policy, threshold and provider; the readiness check above is in the template
60 days
Draft safely
Release-note prompt reviewed through a PR; every draft carries links; first evaluation set of past records built
90 days
Check at scale
Evidence-completeness check runs on every AI change; one tabletop drill done; untracked-change metric reported