An AI system is not governed because a policy exists. It is governed when someone owns it, its evidence is bounded, its exceptions have a destination, and it can fail without taking the business with it.
AI governance is usually presented as a set of principles. Be transparent. Protect privacy. Keep a human involved. Monitor risk. Use AI responsibly.
The principles are reasonable. They do not tell an operator what to do when two sources disagree, a model changes behaviour, a connected tool fails halfway through an action, an employee asks the system to exceed its role, a customer challenges an automated outcome, or the owner is unavailable.
An operating model must survive those moments. For a small or medium-sized company the useful minimum has four parts: ownership, evidence, escalation, failure.
1. Ownership
Every AI system needs four named roles, even if one person holds more than one.
Business owner. Accountable for purpose, accepted risk, value, continued use and affected people. Decides whether the system should exist.
System owner. Accountable for configuration, permissions, versions, connected tools, change control and availability. Keeps the implementation inside its approved design.
Evidence owner. Accountable for source quality, currency, access, retention and correction. A knowledge assistant built on unowned documents becomes less reliable as those documents age.
Decision owner. Accountable for final human judgment where required, exceptions, disputed outcomes and reversals. The decision owner cannot be “the AI team.”
Maintain a system register with one row per deployed system: name, purpose, owner roles, users, affected people, authority level, data categories, approved sources, tools, model or provider, start date, review date, last material change, current status.
A company cannot govern systems it has not identified.
2. Evidence
Generative AI can produce a coherent answer without reliable evidence. Governance therefore needs an evidence model, not only an output policy.
For every system define source scope (which repositories may be used), source authority (which wins when information conflicts), source currency (who updates it and how expiry is recognised), source lineage (can the operator identify which record supported an output), and source access.
That last one matters more than it sounds. Does the user have permission to see the source, not merely permission to ask the assistant? An AI layer must not flatten existing access controls.
Then use four evidence states. Verified: an approved source supports the statement. Declared: an authorised party supplied it, but independent support is absent. Inferred: the system derived it from other information. Unknown: evidence is missing or conflicting.
The output should not present all four with the same certainty.
3. Escalation
Escalation is the route from machine uncertainty to human authority. Four classes.
Clarification. The request or input is incomplete.
Exception. The case does not fit an approved rule.
Approval. The action is inside scope but requires human confirmation.
Incident. The system may have acted incorrectly, exposed data, exceeded authority or become unsafe.
Each class needs a trigger, recipient, response time, evidence bundle, temporary system state and fallback. An incident should not enter the same queue as a missing form field.
4. Failure
The goal is not a system that never fails. That promise is not credible. The goal is a system whose failures are constrained, visible, recoverable, recorded and assigned.
Define the failure modes: incorrect output, unsupported output, stale source, missing source, unauthorised access, duplicated action, partial action, tool failure, prompt injection, provider outage, excessive cost, unexpected drift.
Define the safe state. What happens when the system cannot continue safely? Save a draft without sending. Stop spend changes. Preserve the last approved record. Route work to a manual queue. Revoke tool access. Display source records without generating a conclusion.
NIST’s AI Risk Management Framework treats safe failure, contextual limits, monitoring and documented oversight as parts of risk management, not as extras.
Then test recovery. A rollback plan that has never been tested is a document. Test credential revocation, provider outage, corrupted context, duplicated trigger, a bad model release, an unavailable owner, and restoration of the last approved version.
The operating cycle
Governance is not completed at launch.
Register. Record purpose, ownership, data, authority and dependencies. Assess. Map benefit, consequence, affected people, evidence and failure. Approve. Approve the operating charter, permissions, evaluation and release stage. Monitor. Track quality, escalation, override, incidents, cost and drift. Review. After a fixed period and after any material change. Retire. Remove permissions, export required records, delete unnecessary data, close dependencies.
Systems that are no longer used can retain live credentials and data access. Retirement is part of operation.
Three review cadences
Per run: validation, evidence, authority, action record.
Monthly: error and escalation patterns, adoption, permissions, cost, source currency.
Material change: triggered by a model change, new connector, new data category, increased authority, new user group, new market, incident, or legal change. Do not wait for the monthly meeting after a material change.
What changed in Europe
The EU AI Act entered into force on 1 August 2024 and applies in stages. Literacy duties have applied since 2 February 2025. Following the 2026 Digital Omnibus amendment, providers and deployers must take measures supporting AI literacy among staff and others using systems on their behalf.
Obligations for providers of general-purpose AI models have applied since 2 August 2025. Transparency obligations for certain interactive and generated-content systems apply from 2 August 2026, with a later transition for machine-readable marking by pre-existing generative systems.
An ordinary business AI use is not automatically high-risk under the Act. Classification depends on whether the system falls within the defined categories and context. That is not an argument for having no operating controls. Privacy, consumer, employment, copyright, confidentiality, security and sector duties continue to apply.
Governance should begin with what the system actually does, not with the assumption that “low-risk” means consequence-free.
A minimum approval pack
Before deployment, require a system-register entry, an operating charter, a data and source map, an authority level, evaluation results, an escalation matrix, a failure and recovery test, an operator card, a monitoring plan, and a named retirement owner.
This can be concise for a narrow drafting tool. It should expand with authority and consequence.
How we use it
We treat this four-part model as the common grammar across every division. Who carries the result. What the system may know. Where uncertainty goes. How the system stops and recovers.
Scale, Origin or Merch supply the operating context. We remain responsible for the whole. The decisions that feed it come from a recurring decision map.
Principles describe the company we want to be. An operating model decides what happens on Tuesday when the source is wrong.
If you already have AI systems running and cannot name their owners, start there.
Sources
NIST AI Risk Management Framework.
NIST Generative AI Profile.
European Commission, AI Act implementation timeline.
Regulation (EU) 2026/1744 amending the AI Act.
OWASP Top 10 for Large Language Model Applications.
Current to 30 July 2026. This article provides an operating framework, not legal advice.
