Can AI Be Made Reliable? This Computer Science Professor Says Yes

TLDR

• UC Berkeley professor Dan Klein co-founded Scaled Cognition to build a frontier AI model from scratch to combine generative AI’s flexibility with deterministic controls.
• The startup generates synthetic interactions and uses verifiable reinforcement learning to train APT on actions, information, APIs and business policies rather than language alone.
• Scaled Cognition’s 100% reliability caveat: The guarantee applies to the AI model following explicitly defined constraints.

Dan Klein has spent decades working on artificial intelligence systems that understand language. He has co-founded and sold two startups, including one to Microsoft where he spent four years as a technical fellow.

But the computer science professor at the University of California, Berkeley, also saw a problem that could limit how far generative AI could spread: Businesses wouldn’t entrust important tasks to systems they couldn’t reliably control.

A large language model can be forgiven for writing a thank you note that doesn’t sound exactly the same every single time it is generated. But that won’t work for tasks that require 100% accuracy like getting a bank balance.

Three years ago, months after ChatGPT debuted, Klein co-founded a startup to solve that problem. This June, Scaled Cognition raised $100 million in a Series A led by Khosla Ventures for a total fundraise of $121 million. Recently, the startup unveiled Voice, a text-to-speech product for customer experience that works as a standalone product or with its frontier AI model.

“What was on my mind was, just think of all of the things we could do with AI if you could just trust the system, if you could trust it to do what you told it to do every single time, and if you could trust it to never do the things that you’ve told it to never do,” Klein told The AI Innovator.

“What became very clear to me is that this isn’t something that was going to change by doubling down on existing approaches, that new architectures were needed,” he added. “We started Scaled Cognition to build new technologies exactly to solve that problem of reliability. Our goal, in a word, is super-reliability.”

Scaled Cognition chose customer experience as its first target market because of the high stakes of getting the information wrong. But Klein said they also realized that “every industry considers mistakes to be high cost.” The startup is planning to expand its target market accordingly.

Harnesses come with complexity

Klein said the industry has been trying to solve that problem largely by using harnesses, or controls around models with specific rules. He believes this adds complexity, latency and costs to the system while constraining the performance of AI intelligence.

Klein thinks it’s better to integrate those controls directly in the model architecture. So the startup built its own model from scratch.

Scaled Cognition’s model is called Agentic Pretrained Transformer, or APT. The startup said it didn’t take an existing large language model, such as Claude, and then fine-tuned it for its purposes. Rather, it built the frontier AI model from scratch. (The model’s parameter size wasn’t disclosed.)

“The idea behind our company is building models that you can really rely on in these enterprises contexts where it has to be right over time, where it’s taking an action, where if that action is wrong, there is going to be a cost,” Klein said.

How is reliability ensured? A conventional language model primarily predicts tokens – the pieces of words that form its response. Klein said Scaled Cognition built APT to explicitly model two additional things: information and actions. That distinction allows companies to define policies over those structured objects.

Instead of merely telling an AI agent, “Always obtain mortgage rates from this database,” the system can define the database as the required source for that information. The language surrounding the transaction can remain flexible while the source requirement is deterministic.

Klein compares the idea with a compiler, the software that translates computer code into instructions a machine can execute.

Scaled Cognition claims 100% reliability, but Klein caveats it somewhat.

APT can still misunderstand a customer or encounter a situation that isn’t covered by a policy. The 100% guarantee applies to whether the AI model will always adhere to explicity defined company rules. “For the things that are 100%, it’s 100%,” Klein said.

Where’s the data?

Building a different architecture created another problem: finding data to train it.

The web contains enormous amounts of language, but relatively little data showing the full sequence of actions an AI agent should take while operating under business policies.

Scaled Cognition decided to manufacture the data itself.

“We generate our data internally through simulation,” Klein said. “We have a verifiable RL (reinforcement learning) pipeline.”

The company creates simulated interactions that include a customer, information, APIs, business policies and possible actions. Those interactions produce trajectories – sequences showing how an encounter unfolds and what the agent does along the way.

The approach borrows an idea from some of AI’s biggest advances.

A chess program can play repeatedly and determine who won. Software can generate computer code and run tests to see whether it works. A mathematical proof can be checked by a formal verification system.

“The core idea in our training is to extend that sort of verifiable framework to agentic interactions,” Klein said.

Scaled Cognition applies reinforcement learning to the simulated trajectories, training the model on behavior that succeeds under the conditions established by the simulation.

Consider a simulated bank customer asking to transfer $500 between accounts. The environment can specify the account balances, authentication requirements, permitted APIs and transfer rules.

If an agent authenticates the customer, selects the correct accounts, calls the required transfer API and reports success only after the transaction succeeds, the trajectory can be verified.

If the agent tells the customer the transfer succeeded without making the API call, it fails.

That is substantially different from asking another AI model whether the response sounds convincing. “When the model makes a decision, it makes a decision about actions and information rather than tokens,” Klein said.

This reliability is what Scaled Cognition believes people will insist on.

“What’s next for us is really continuing to add more and more products and surface area that bring that level of super-reliability to more and more areas of AI,” he said.