Who’s Responsible When AI Misfires in Enterprises?

Bill Gates is the latest tech mogul to raise the alarm on the potentially dire consequences of artificial intelligence, following in the footsteps of other AI leaders in calling for an immediate slowdown and stricter regulation in the development of the technology.

But how close are we to seeing some of these risks materialize, and are these concerns reflected in what businesses are experiencing today? 

Such concerns were recently amplified by AI agents going rogue, OpenAI halting training of its most recent frontier models, and the signing of a voluntary AI safety pact by leading AI tech Executives and the Trump administration. Recent research we commissioned suggests there is certainly valid reason for apprehension – but also a safer path forward.

Our findings showed an alarming 60% of U.S. companies using AI said the technology is developing faster than they can govern it.

Also worrying, business leaders can’t agree on who is responsible when AI systems produce harmful or inaccurate results. It found that nearly a third (31%) felt it should be shared between the organization and the vendor. Meanwhile, a quarter (26%) believe the organization using the AI should bear the responsibility, and a fifth (20%) think it lies fully with the AI vendor. The rest say the user or regulators should take the blame.

Call me old-fashioned, but shouldn’t absolute trust be the gold standard in any relationship, even with AI?

This fractured approach to oversight poses a major threat because when accountability is split among too many stakeholders, true ownership vanishes entirely. Should a critical AI breakdown occur, this lack of clarity inevitably leads to finger-pointing, stalled interventions, and severe regulatory risk at the exact moment that rapid, decisive action is required. Modern compliance regulations such as the EU AI Act will not accept a web of shared confusion as a valid legal defense.

What also stood out from the survey was that only a quarter (26%) of business leaders using AI in their organization say they completely trust it to produce accurate outputs, while just 22% completely trust AI systems to operate without introducing unacceptable risks.

Call me old-fashioned, but shouldn’t absolute trust be the gold standard in any relationship, even with AI? The survey highlighted the uncomfortable reality that organizations are scaling tools they do not fully trust, without a clear agreement on who answers when that trust proves misplaced.

How AI governance helps

Effective governance plays a key role in helping organizations realize greater value from AI. In the U.S., our survey showed that 85% of businesses have an AI governance framework in place and 83% say it has made AI initiatives more successful.

Efficiency is the core driver for AI implementation, but proper governance drives success. When done correctly, governance can ensure both speed and safety, giving your AI projects the structure they need to provide the results you want. This is the true differentiator that gives teams the confidence to move AI out of pilot purgatory and into production, because the controls, accountability, and oversight are already in place.

The path forward does not require slowing down innovation; it requires closing the distance between adoption and accountability throughout the entire organization.

Here are some key strategies to consider:

1. Establish a formal AI governance policy that lays out the organizational governance structure and technical measures that enable trustworthy AI, including fairness, accountability, transparency, and privacy related to the operation and outcomes of AI systems.
2. Introduce algorithmic risk assessments before deployment that encompass AI risk classification, identify known, foreseeable, and residual risks based on their severity, likelihood of occurrence, and detectability, and institute controls to mitigate the adverse impact of AI system outputs.
3. Make data governance the foundation of AI governance. It has been said that “bias is machine learning’s original sin.” AI systems are probabilistic models, and their accuracy and reliability depend on the provenance, accuracy, completeness and representation of the training and testing data pipelines. Organizations must monitor for data quality problems, bias, drift and other changes that could affect model performance or introduce unintended outcomes.
4. Embed cybersecurity and privacy into the AI lifecycle. AI creates new attack surfaces such as data exfiltration and model inversion attacks that can amplify existing security and privacy risks. AI governance therefore needs to integrate with existing cybersecurity, privacy, and enterprise risk-management programs rather than operate as a separate discipline.
5. Design meaningful human oversight into AI systems, including effective human-to-machine interfaces for humans to monitor performance, intervene and override adverse results or stop an AI-generated action.

If there was only one takeaway I could leave you with, it’s that AI governance should not be viewed as the enemy of innovation. It may be one of the things that makes innovation possible at scale.

Organizations that establish clear accountability, build practical safeguards and ensure employees understand their responsibilities can move faster with greater confidence. The goal is not to slow AI down, it is to make sure that when AI moves fast, the organization knows who is in control, who is accountable, and what happens when things go wrong.