FICO CAO: Building Smaller, High-Performance AI Models Responsibly

TLDR

• Smaller, domain-specific AI models can outperform general-purpose LLMs in regulated industries while providing better control, transparency and lower costs, according to FICO Chief Analytics Officer Scott Zoldi.
• Rather than relying on RAG or fine-tuning large models, organizations should build specialized models trained only on relevant data and use curated examples to teach the desired outcomes.
• FICO’s “trust score” approach uses a second AI model to audit the first, giving risk leaders a way to measure, control and document AI decisions before they are deployed.

As frontier AI companies race toward building ever-larger models, FICO Chief Analytics Officer Scott Zoldi is making the case for moving in the opposite direction.

Speaking at the recent AI Hot 100 Summit in New York and in a separate interview, Zoldi argued that organizations in highly regulated industries such as banking should abandon the notion that a massive general-purpose AI model can safely power critical business decisions.

Instead, he advocated building smaller, domain-specific language models trained on carefully curated data and monitored through what he calls “trust scores.”

“The open models that are out there today and the global models that are out there today don’t meet the standards of responsible AI,” Zoldi told The AI Innovator, which was a conference media partner. “They need to get to robust AI, explainable AI, ethical AI and auditable AI to ensure that the basic data science practices we’ve been doing for decades are not violated.”

The approach represents a sharp contrast to the prevailing enterprise AI strategy, where companies often connect proprietary data to general-purpose large language models and then constrain them through retrieval systems, fine-tuning or prompting techniques.

According to Zoldi, that model introduces significant governance problems.

Financial institutions have used AI for decades and operate under strict regulatory requirements governing fairness, explainability and risk management. When financial institutions deploy AI systems they did not build themselves, he said, they often cannot answer basic questions about how the models were trained, whether the data is representative, or whether hidden biases exist.

Another advantage is cost. If organizations build and run the models themselves, they may be able to avoid per-token charges from third-party AI providers or significantly reduce inference costs.

How to build a focused SLM

Zoldi’s solution starts with building a focused language model from scratch.

Rather than training on the broad internet, Zoldi proposes restricting training data to a specific domain. In financial services, for example, the model would learn only information relevant to banking, lending, fraud, compliance and related financial topics. Information unrelated to those tasks would be excluded.

“If you’re a financial services organization, you do not need a model that’s built on Spice Girls lyrics. You do not need a model to tell you how to change a tire. You need a model that only knows about financial services,” he said.

The idea is straightforward: Every piece of irrelevant information added to a model creates additional opportunities for hallucinations, confusion or inaccurate responses.

To create these focused models, FICO first defines a domain and then filters large datasets to identify documents relevant to that domain. “Take a small sample and ask a decoder model to look at those documents,” he said. The decoder model assigns each document a score from one to 999 indicating how relevant it is to the domain.

Next, an encoder model is used to sift through the organization’s available data and identify the small percentage that is relevant to the task. That filtered dataset becomes the development dataset used to train the model.

For businesses balking at building these models, Zoldi is reassuring. “That’s not hard. Get your PyTorch,” he said, adding that “you don’t need a lot of GPUs.” When he built his first domain model at FICO, he said he only used two nodes equipped with Nvidia H100 chips. “It’s not a big investment.”

Don’t just rely on RAG or fine-tuning

The resulting models, despite being dramatically smaller than frontier AI models, can outperform generic LLMs on financial tasks because they are trained exclusively on relevant information. “Even out of the gate, these models are more performant than even the most current (general-purpose) models, because they’re focused just on this industry,” he explained.

Zoldi argued that relying on a generic LLM constrained by Retrieval Augmented Generation (RAG) or fine-tuning techniques may produce less specialized results. (RAG gives the model access to a company’s data so it can look up relevant information before answering a question. Fine-tuning further trains the model on domain-specific data.)

“A large commoditized model has seen the entire totality of all the data known to mankind,” Zoldi argued. “So how do we think that we’re going to persuade it to ignore everything that it has learned? By prompting correctly? How do you have confidence in that?”

That’s why he prefers a domain model. But don’t build one model that can do many tasks. Rather, build multiple models that are each focused on one specific task, he said. Tailor each task to a use case, such as customer compliance. For example, banks are required to respond in certain ways if clients are having issues. If a client tells the bank they just got divorced, the bank may be obligated to offer hardship help or discuss hardship options.

Then create seed data: a carefully reviewed set of examples.

But Zoldi warns against training the AI model on past data because it will learn from errors. For example, if a bank has 100 human underwriters, some might have approved loans that shouldn’t have been approved and some may have wrongly denied loans. If a company uses this historical data to train the AI model, it will learn those mistakes.

Instead, create a small set of examples that show the correct way to handle a situation. This seed data becomes the foundation for training the AI. The company can then generate additional synthetic examples based on those cases so the model can learn the organization’s desired practices rather than repeating old mistakes, according to Zoldi.

Trust scores and a ‘knob’ for the CRO

But how can organizations trust the AI model’s output? Zoldi’s solution is the trust score

The idea is to build a second AI model that audits the first model. A separate group of business experts, lawyers, compliance officers and risk managers would create what he called “knowledge anchors.” These are examples of how the company believes certain situations should be handled.

Build a separate model to analyze those guidelines using machine learning techniques such as latent Dirichlet allocation, a statistical method for identifying patterns and themes in text. The AI’s responses are then compared against those ‘knowledge anchors.’ If a response is closely aligned with the organization’s expectations, it receives a high trust score. If it does not, the score falls accordingly.

Chief risk officers (CROs) can choose how much freedom they want to give an AI system. In lower-risk applications, they may tolerate occasional hallucinations. In higher-risk decisions, they can tighten trust thresholds and require much stricter alignment with company policies. For example, a company may only use AI-generated responses 70% of the time if it sets a very high trust threshold.

“The CROs need the knob,” Zoldi said. “Otherwise, they have no way to control the risks of generative AI.”

Organizations do need to refresh seed data and update their models several times a year as regulations and business processes evolve – and it’s less laborious than one would think. “It’s not scary updating these models with just two nodes of H100s,” Zoldi said. “It takes four days. It doesn’t take four months. It doesn’t take $5 million.”

“If you’re a financial services organization, you do not need a model that’s built on Spice Girls lyrics.“

As enterprises adopt agentic AI systems, he sees trust and transparency becoming even more important in the face of autonomous bots.

FICO has developed what Zoldi called AI blockchains to document how models are built, tested and monitored. The company is extending that concept to agentic AI through what he describes as a blockchain-based record of every decision made by a collection of AI agents.

Zoldi referred to the resulting audit trail as “decision DNA.”

The goal is to create a permanent record showing which agents participated in a decision, what information they used and how the final outcome was reached. Such transparency could help organizations investigate errors, explain decisions to regulators and establish accountability in increasingly autonomous AI systems.

“This is the way we need to be thinking about how we will work with generative AI in ways that solve business problems where we have control of the tenets of responsible AI,” Zoldi said.