AT&T saves up to 90% of AI costs by using open models, and plans to eventually run 70% to 80% of its AI workloads on them, according to The Wall Street Journal.
Open models currently account for about 25% of AT&T’s AI usage, Chief Data and AI Officer Andy Markus told the newspaper. Open models are AI models that a developer or company can download, run and often modify themselves rather than accessing them only through the model builder’s hosted service.
The telecom giant’s experience is bad news for top U.S. AI companies – such as OpenAI, Anthropic and Google – whose AI models are mostly closed and typically more expensive. While many top Chinese models are open, there’s some effort to close them.
AT&T’s use of open models, including Chinese models, helps save millions of dollars as it consumes about 45 billion AI tokens a day. The company has built an AI gateway that evaluates factors including cost, speed and expected output quality and automatically routes tasks to the model best suited for the job. The system can even switch models during a multi-turn interaction, Markus wrote in a July company blog post.
At AT&T, only a relatively small percentage of tasks require the capabilities of the most advanced AI models, he said. Less expensive models can perform many other jobs without sacrificing the performance AT&T needs.
Accuracy remains the deciding factor. AT&T tests models against one another and selects them based on whether they meet the requirements of the particular business application, Markus told the Journal. The company said its AI initiatives are generating a five-fold ROI within the year.
AT&T already has more than 1,000 internal AI uses spanning legal, finance, customer service, field operations and its telecommunications network. Open models entirely handle the company’s summarization and analysis of customer-service call transcripts, while a telecom-specific model called OTel supports agents that can help identify the causes of network problems.
The company has also been developing more specialized telecom models. AT&T said its newer OTel 2.0 model was post-trained using more than 400 billion tokens and is designed so telecom workloads can run on hardware either on premises or in the cloud, Markus wrote.