Skip to main content
Dark green background, "Weak Application Security Can Cost You Millions," 3 slanted images of fingers pointing to digital locks, and a "Learn the Basics" button
Who Actually Owns AI When It Fails?Governance & Controls
5 min readFor Compliance Officers

Who Actually Owns AI When It Fails?

About six months ago, compliance officers began asking tough questions. They'd been told they're now "responsible for AI governance" without any additional resources or clear guidelines. The stakes are high: When an AI system makes a discriminatory hiring decision or approves a fraudulent loan, who faces the regulator?

Here's what we're hearing most often, and what you can do about it.

Q1: "Am I accountable for AI systems I didn't even know we were using?"

Yes, if those systems operate within your responsibility.

In UK financial services, the senior managers and certification regime makes this clear. The UK Financial Conduct Authority and the Bank of England's Prudential Regulation Authority decided that AI doesn't need its own prescribed responsibility. Instead, if you're the senior manager accountable for retail lending, you're accountable for AI used in retail lending.

This isn't unique to UK finance. Regulators investigating an AI failure follow the decision trail: Who raised the risk? Who oversaw the function? Who reviewed the output? That trail often leads through compliance and risk, whether formally assigned or not.

The practical step: Map every AI system to a named senior manager who owns the business function it supports. If you can't name that person for each system, you've found your first gap.

Q2: "Can't we just wait for the EU AI Act deadlines to figure this out?"

No. The EU AI Act high-risk obligations for stand-alone systems are deferred to December 2027, and for AI embedded in regulated products to August 2028. But Section 80 of the Data (Use and Access) Act 2025 is already in force in the UK, giving individuals rights to transparency, human review, and the ability to contest automated decisions in hiring, credit, and insurance. The EU AI Act's Article 50 transparency obligation is also live.

You're not waiting for a starting gun. Some requirements are already in effect.

Q3: "What does 'reasonable steps to oversee' mean if I'm not technical?"

It means you can show you asked the right questions and documented the answers.

You don't need to audit the model's training data yourself. You need to know who did, when, what they found, and what action was taken. Document that someone reviewed the system's outputs for bias or error, assessed its intended use, and escalated concerns.

If a regulator asks what steps you took, they'll want evidence of an oversight process, not proof you understand technical details. The question is whether you treated AI as a material risk requiring regular review.

Create a standing agenda item for AI system reviews in your risk committee meetings. Document who presented, what was discussed, and what follow-up actions were assigned. Those minutes are your evidence.

Q4: "We have an AI policy. Isn't that enough?"

A policy states the rules. It doesn't answer who was accountable when those rules weren't followed.

After an incident, regulators don't ask to see your policy first. They ask who knew what, when they knew it, and what they did about it. A policy that says "all AI systems must be reviewed for bias" is only useful if you can produce a risk register showing which systems were reviewed, by whom, on what date, and what the outcome was.

Common gaps in AI governance reviews include: no named owner for specific systems, no documented risk register with an owner attached to each entry, and a review cadence that exists on paper but produces no substantive discussion. An AI update tacked onto the end of a routine meeting with no real minutes isn't evidence of oversight.

Q5: "How do I assign accountability when I don't even know all the AI tools we're using?"

Start with an inventory, but don't wait to finish it before you assign owners.

The inventory question is: What AI systems are in use, in what functions, making what kinds of decisions? The accountability question is: For each system, who is the named person responsible if it produces a harmful output next week?

You can answer the second question for the systems you know about today. Assign a senior manager owner to each one, with a defined review schedule. Then expand the inventory and repeat. Waiting until you have a complete list before assigning any accountability just extends the period where nobody owns anything.

If you're in a regulated function like lending, underwriting, or hiring, start there. Those are the areas where automated decision-making obligations are already live.

Q6: "What happens if I can't prove I was overseeing a system that failed?"

You become the person regulators investigate for failing to take reasonable steps.

Under the senior managers and certification regime, that can mean personal sanctions. Outside that regime, it still means you're the named individual who has to explain why oversight didn't happen. The organization may face penalties, but you face the reputational and professional consequences of being the person who was supposed to be watching and wasn't.

This isn't theoretical. When an AI system produces a discriminatory outcome or a compliance failure, the question isn't just "what went wrong?" It's "who was accountable for preventing this, and why didn't they?"

The answer can't be "we didn't think anyone needed to own it specifically."

Where to go for more

If you're in UK financial services, review the senior managers and certification regime guidance on accountability for business areas. If you're using AI for automated decisions in hiring, credit, or insurance, review Section 80 of the Data (Use and Access) Act 2025 and the EU AI Act's Article 50 transparency requirements.

For a broader framework, the NIST AI Risk Management Framework provides a structure for identifying, assessing, and managing AI risks without requiring deep technical expertise. ISO/IEC 42001 offers a management system standard for AI governance that maps to existing ISO/IEC 27001 processes.

But the most practical step is internal: Before your next AI system goes live, ask who the named person is who'll be accountable if it goes wrong, and whether they can prove they were watching it.

Promotional banner highlighting failures found in PCI audits and how to spot the gaps

You Might Also Like