Imagine if a newly hired employee arrives for her first day of work, eager and ready to start, but lacks an ID badge, access to company systems, knowledge of HR policies and the permissions she needs to do her job. She might be brilliant, but without the infrastructure that lets her operate inside the company, she won’t accomplish much.
AI agents require something similar. As enterprises race to deploy AI agents across their organizations, they will increasingly need components that govern, connect and manage them, according to Jim Rowan, Deloitte’s U.S. head of AI.
Need more clues? Ask the Sherlock chatbot in the lower right corner to summarize this story, explain technical concepts or answer other questions.
Nearly four years after ChatGPT’s debut, companies are putting their focus beyond large language models and toward the operational challenge of deploying AI agents that interact with enterprise applications, corporate data and each other.
To make that work reliably, organizations need what Deloitte calls an “agent harness” — a management layer that surrounds AI models with governance, memory, connectors and operational controls.
“The models are very good at providing the brain,” Rowan said in an interview with The AI Innovator. “The harness itself is how the model can start to actually do activities and do work and connect to data, memory, applications, user experiences … all the things that make knowledge work happen at scale.”
Rather than replacing existing enterprise architecture, Rowan described the harness as its next evolution. As AI models became more capable, companies realized employees needed additional capabilities surrounding those models before they could be deployed broadly across the business.
“The models got smarter,” he said. “But we also realized that users needed things around the models to be successful.”
A typical harness includes a user interface, governance and policy controls, memory, connectors to third-party systems and protocols that allow multiple AI agents to communicate with one another. Different software vendors will implement those capabilities differently, Rowan said, and enterprises are likely to combine proprietary and open-source components.
Harness vs. orchestration
Rowan distinguishes the harness from the broader AI orchestration layer, although the two concepts overlap. The orchestration layer acts like the traffic controller, coordinating, routing and managing tasks and workflows across potentially hundreds or thousands of agents.
“The harness is part of the orchestration layer,” he said.
That orchestration layer, he argued, will become increasingly important as organizations begin deploying fleets of AI agents. It also raises new questions about how enterprises should manage personal AI agents created by employees alongside enterprise-managed agents.
Despite growing interest in AI agents, Rowan believes the market remains in its infancy.
“We’re still early,” he said.
Current platforms are improving their ability to monitor agent usage, track performance and enforce governance, but important gaps remain. Compliance capabilities need to mature, interoperability between agents remains limited and enterprises still lack intuitive ways to visualize and manage large fleets of agents, he said.
Perhaps the biggest challenge is governance.
Many organizations still rely on policy documents and employee training to govern AI use. Rowan said those policies ultimately need to become machine-readable and embedded directly into enterprise software so AI agents automatically follow changing regulations and company rules.
“We have policies that are Word documents and text files,” he said. “Getting those codified and integrated into applications … that’s a huge undertaking.”
He added that companies have yet to fully address how they will retain, delete and secure the growing volume of operational data generated by AI agents.
The models got smarter. But we also realized that users needed things around the models to be successful.
Enterprises also face strategic decisions over who should own the broader orchestration layer that manages AI agents.
Some organizations may rely on cloud providers or software vendors, while others will build portions themselves. Companies can choose to adopt different approaches, particularly for AI systems supporting proprietary business processes.
“The closer you are to the heart of the business, the core processes that make you different, the IP that makes you unique,” he said, “you’re going to want to own more and more of that.”
Ultimately, Rowan believes the technology itself may evolve faster than its terminology.
“I think it’s going to get really blurry over the next six months between what is a harness and what’s an application,” he said. “Do we drop the harness terminology in another year?”
Regardless of what the industry eventually calls it, he said enterprises will need a centralized way to connect, govern and monitor AI agents if they hope to move beyond isolated pilots and deploy agentic AI at scale.
Be First to Comment