Artificial Intelligence1 min read

Amazon saves millions by replacing Sonnet 4.6 with Strands Decider

By · Published by Everything Blog

In short

The new Strands Decider E4B Gemma 4 implementation matches or exceeds previous production benchmark performance while eliminating text-generation overhead. Because the model scores probabilities for specific options rather than returning complex JSON payloads, token consumption decreased from 6,724 to 2,010 tokens per ticket. Furthermore, average operational latency improved by 78%, dropping response times down to 897 milliseconds. The entire pipeline runs via Amazon SQS, AWS Lambda, and SageMaker ml.g5.xlarge instances.

Key points

  • Transitioning from generative language models to dedicated decision models can drasticall…: Transitioning from generative language models to dedicated decision models can drastically reduce recurring enterprise token costs.
  • Scaling down to network-isolated Amazon SageMaker ml.g5.xlarge instances slashes expenses…: Scaling down to network-isolated Amazon SageMaker ml.g5.xlarge instances slashes expenses down to $500 per million tickets.
  • Specialized decision architectures can deliver faster operational runtimes while matching…: Specialized decision architectures can deliver faster operational runtimes while matching or exceeding the baseline accuracy of large generative models.

Sources checked by our analysis pipeline: 1

For a while, Amazon and AWS teams relied on Claude Sonnet 4.6 to label tickets in an internal system without breaking the bank.

Sonnet 4.6 fit the goldilocks choice of balancing intelligence with cost. Considering it dealt with one of the world’s largest internal ticketing systems, the nine month old model did well. However, two Amazon engineers recently swapped out Sonnet 4.6 for Strands Decider, a decision model we launched a month ago, that performed just as well at 98% less cost. This change will save millions of dollars of token costs in a year.

Carr and Mehul, who work on the internal ticketing system, walk through how this happened below.

Replacing an LLM’s job with a decision model

We had Sonnet 4.6 labeling tickets at enormous scale. The tickets could be customer escalations, capacity requests, automated alarms, security findings, or anything else across the company that needed to fit within 20 category types. The agent ran with a long system prompt on every ticket, generating JSON and reasoning against a fixed set of labels.

A decision model can’t generate…

Artificial Intelligence