In Amazon’s recent Q2 earnings call and follow-up LinkedIn post, CEO Andy Jassy unveiled some jaw-dropping statistics about the results of using its coding and conversational AI assistant – Amazon Q Developer – internally.
- The average time to upgrade an application to Java 17 plunged from 50 developer days to a few hours. This saved the company around 4,500 years of work.
- In six months, more than 50% of production Java systems were upgraded.
- The upgrades lowered infrastructure costs and enhanced security, leading to $260 million in annual efficiency gains.
Anu Sharma, AWS director of Amazon Bedrock experiences and tools, helped create the Amazon Q Developer on its Bedrock AI service platform. In this interview with Deborah Yao, editor-in-chief of The Innovator, Sharma discussed the development of the AI coding assistant and said AWS is developing ‘validators’ for their clients. She also shared her views on what’s ahead for agentic AI.
The AI Innovator: Could you please introduce yourself and describe your role at AWS?
Anu Sharma: I lead product for Bedrock Tooling and Experiences. This includes Bedrock capabilities including agents, guardrails, knowledge bases, some of the things that are finding the highest degree of adoption across AWS and we’re really excited about customer responses to those.
I also own a second piece called Bedrock Experiences. Some of that includes Bedrock studio, which some of our customers are already using and is in preview at the moment. Super excited about what we’re working on. Happy to get into any details if you’d like.
Can you tell me about Bedrock?
Bedrock is our flagship generative AI service that enables customers to build a foundation that enables them to build generative AI applications in a more secure and private way without compromising their own enterprise data. It’s part of the three-layered stack that we have established in AWS.
At the bottom of the layer is the foundational infrastructure layer that includes our EC2 instances including GPU instances, the best in class, as well as our Trainium and Inferentia instances and infrastructure as well as Sagemaker that you would use to build some of these valuation models and world class models. At the middle of the layer is where Bedrock sits and enables customers to seamlessly build applications using this infrastructure.
And at the top of the layer, the third layer of the stack, is the application layer where you’ll find Amazon Q, Q Developer and Q Business, Q in Connect and various other applications that enable customers to improve their end user experience, their employee productivity. All of those are built on Bedrock as Bedrock is built on some of our foundational infrastructure.
Can you tell me more about Amazon Q on Bedrock, including the underlying language models, parameters, whatever you can share?
All of our generative AI applications are built on Bedrock because it accelerates our ability to build those applications because of the security foundations that is already laid out for us. As part of that, it enables us to build an agentic framework. So Q Developer has built up two agents, Q Developer Agent for software development and Q Developer Agent for code transformation built on Bedrock agents.
The foundation models that it uses underneath include a range of ones that are best suited for the use cases that we’re trying to serve customers. These can be from the code completion in the editor to long range, long horizon tasks which require larger models. Some of the models will be lower latency and smaller models. So there’s no one model that rules them all. That is the second piece that Bedrock brings to us is the ability to try and evaluate more models and choose the right ones for the use case.
Did you exclusively use Amazon foundation models to develop Q?
It’s a combination of our in-house developer model, developed models, as well as third party models. We use the best-in-class models that are available and are brought to us on Bedrock by our third party partners as well as our internal teams.
Your CEO Andy Jassy released some statistics about usage and savings with Q Developer that were pretty impressive. Were you expecting those results?
Yes. In fact, I was partly involved in delivering that result and building that service from scratch. [For instance,] upgrading over 30,000 Java applications and saving over 4,500 in developer years. This to me was one of the most exciting projects to be part of. We knew from the beginning that this was something exciting for us to move the needle on, to give our developers back their time, which they tend to spend on upgrading software. And it is essential for us to upgrade software to maintain the security bar and performance on behalf of our customers. Because the bar is high, this tends to take away time from us developing new features for our customers.
Our ability to save this time now means that we’re able to innovate faster for our customers. And developers also love being part of … and investing in the creative part of their jobs rather than working on some of the undifferentiated heavy lifting that they have had to do in the past.
How do you see your clients leveraging Q developer? And how is it different from other genAI coding assistants out there?
Several of our customers have written to us and told us that it is one of the best in terms of acceptance rates that they are seeing in the industry. For example, British Telecom has given us 37% acceptance rates in the work that they have done with their engineers. National Australia Bank is seeing 50% acceptance rates for the work that they’re doing. So we know that it is best-in-class in terms of how fast it enables developers to write code.
But that’s not where it ends. We believe that we are helping redefine productivity overall in the day-to-day life of a developer. We believe that with world class security scanning, we enable customers to write code that is by definition secure.
And thirdly, our work on agents is probably the most differentiated. You’ll see on the SWE-bench (coding benchmark) that Q Developer Agent for software development is at the top of the leader board in results and evaluations where we scored the highest in terms of the completion rate of the tasks that we took on. These are world-recognized evaluation and benchmarks. We are excited that we are able to deliver this kind of world-leading performance.
How exactly does Q Developer help users as they build generative AI applications?
Q Developer helps at every stage of the software development lifecycle. It helps you recognize and design the right solution, so you can have a conversation in your IDE (Integrated Development Environment that helps developers code more efficiently) with what you want to build and define a high level description of what your application could be. And it will give you a step-wise plan on how to implement it and then you can convert that into code and write the code.
As it writes the code, you can also select pieces of the code and say, ‘Explain this part of the code to me.’ And if you happen to have written existing code and want to debug part of that code, you can select that piece of code and say, ‘Q, debug this for me and or optimize this for me.’ So it is not only part of writing new applications, it can help you resolve and improve the quality of your existing applications.
There’s a second piece to it. It also helps you take this application to production. It doesn’t just stop with the part that our developers say they spend less than 30% of the time on which is development. It helps them put it into production and make sure it is operating using the best practices AWS has developed over the last 17 years. Helping our customers recognize the best practices implicitly and write code from Day One that is both secure and well-architected is our objective. And that means they also have to spend less time operating it.
So closing the loop, it not only helps with the 30% that customers write code on but also on the part that they send to operate the application.
How are you addressing hallucinations, security and privacy issues for Bedrock clients?
At the administrative level, we enable folks who are building those agents to also implement guardrails. Those guardrails can help customers filter out toxicity or PII (Personally Identifiable Information) or bias from Day One. And this comes totally built out of the box and they don’t have to work through how to filter it and build models that will detect such needs for keywords or sentiments that need to be filtered out.
And finally, we also work with third party partners and model developers in detecting abuse, which is automated to the extent that we don’t have to look at the customer’s responses and what is being filtered. We automatically have systems to detect abuse and block that.
How do you see the role of AI agents evolving in the next 5 to 10 years?
Today, we find agents are largely interactive. You can have a conversation. You can develop high-level plans like itineraries and travel, birthday parties, and so on.
Where we see agents developing are in two dimensions. One is that most enterprises have workflows that are typically hand-held and managed by individuals that are making sure that the ‘trains run on time.’ These are automated workflows that agents can both learn from humans, and aid them in – and call for their judgment only when needed, rather than having people do undifferentiated work.
The second area where it’s developing is a goal-seeking agent, which works not just on established SOP but we provide it a goal and it helps discover the different kinds of solutions and works through what is good, or the trade offs involved between different options, and finds the right way or at least a few optimal ways to achieve that objective. And that could be as broad as research.
So for example, as a customer service representative, I could provide the data on past cases and it could tell me what are the biggest patterns, how do I resolve them and how do I in fact convert – bring it back to the automation agents to automate some of those processes so the (representatives) don’t have to spend time automating it. They can spend time talking to customers.
As agents become more autonomous, what safeguards are AWS implementing to make sure they function correctly?
Today, we have guardrails. The second piece that we’re continuing to build are validators. This includes some advanced functionality that we have built internally at Amazon called automated reasoning, which helps customers not only recognize what they have built in the past but convert it into provable outcomes and validated correct results to the extent that they don’t want to share anything wrong with their end users and customers. That’s just an example of the kind of validator that we value as correctness for our agents and turning this back as to why this is relevant to us.
They are highly reliable, they are cost efficient, and easy to use. Those are the three dimensions – reliability, ease of use and cost efficiency – that we are trying to push the boundaries, in terms of what agents are capable of doing.
As agents become more advanced, how does their relationship with humans change or evolve?
I can see an agent working as a way for me to automate the things that I know must happen and I could have done it myself, but I just delegated to the agent because I want somebody else to do the tasks for me. But there’s another kind of agent where in fact that agent, goal-seeking, could come back to me and ask me what I prefer in terms of what direction and search space it should seek. And that is a very human interaction, not just in automating my work, but making me more creative. So I find that there are multiple dimensions in which I find agents adding to human life.
Anything else you’d like to tell readers about agents and AI that we didn’t discuss?
I feel like most technologies are over-hyped in the short term and under-hyped, underappreciated in the long term. And I guess there is a little bit of hype today that I think could hold us back on creating real value for our customers and end users.
And this is where I feel like if we focus on the long term and the ability of our customers to deliver real value, I think the way we operate, the way applications are built, the way software is built and the way commerce is conducted will create a new kind of agentic commerce and marketplace where we will be able to not only scale applications and services and businesses that we have never built or done before, but also create new kinds of businesses that we haven’t yet seen.
