News · 2026-11-10

Amazon describes replacing a general AI model with a ticket decision model

Amazon says it moved an internal ticket classifier from Claude Sonnet 4.6 to a specialized Strands Decider model that scores category choices directly. Its published comparison estimates much lower model-serving costs, although the data needed to independently confirm annual savings and ticket accuracy are not public.

Key facts

• The workflow assigns tickets among 20 categories.
• Amazon’s modeled comparison is about $500 versus $22,200 per million tickets, under its stated assumptions.
• The production case study describes a shadow comparison and a 45-second rollback.
• Primary source: the Strands case study.

A general chatbot can read a ticket, reason about a taxonomy, and write a structured answer. That flexibility is useful, but if the actual job is always to choose one of a known set of labels, generating a long answer may be unnecessary. The Decider approach resembles a multiple-choice examiner: it scores the available options rather than writing an essay before selecting one.

The architecture notes describe a decision head attached to a pretrained model. The public model card describes an adapter and readout head. The case does not establish that the model was trained on Amazon’s private ticket labels. Download size and runtime video-memory requirements are not established in the reviewed evidence.

The engineers say their selected model “beat” Sonnet on a separate set of tickets. They do not disclose the test size, label distribution, ground truth, results, or whether the threshold was tuned on separate data. In the described dual-run period, Sonnet’s answer was applied while Decider was compared with it. Agreement with the old system is useful for finding differences, but it cannot by itself show which system is right.

The cost comparison likewise needs a denominator. The case uses list prices and assumes half of the instance’s theoretical throughput for headroom. Its roughly 44-fold modeled difference is not an audited bill. At that published difference, $2 million in gross annual savings would require roughly 92 million tickets a year before other costs; actual volume and utilization are not disclosed.

The new workflow also changes its inputs: long tickets lose their oldest comments when they exceed the 4,096-token window, and category descriptions are compressed. The team reports that simply reordering JSON keys changed probabilities by up to 0.19 and says it sorts keys to stabilize the representation. That fixes a known source of variation, but the public report does not show broad tests for missing fields, reordered comments, or drift.

In the Hacker News discussion, practitioners suggested conventional classifiers as another possible solution. Those systems may be fast and inexpensive, but need suitable labeled examples and maintenance. The case illustrates the appeal of narrowing a general model’s job; it does not establish that this particular model is the cheapest or most accurate available option.

Primary source, verified: read the paper →

Ground Truth. (2026, November 10). Amazon describes replacing a general AI model with a ticket decision model. Ground Truth. https://groundtruth.day/news/amazon-ticket-classifier-strands-decider.html

@misc{groundtruth:amazon-ticket-classifier-strands-decider,
  title  = {Amazon describes replacing a general AI model with a ticket decision model},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {nov},
  url    = {https://groundtruth.day/news/amazon-ticket-classifier-strands-decider.html}
}

Topics: enterprise · classification · costs