cascades

Model Routing and Cascades: Spending Frontier Money Only Where It Buys Something Lesson

Model routing sends each request to the cheapest model that can handle it, and cascades escalate to a stronger model only when a cheaper one is not confident, which cuts serving costs substantially because most production traffic is easy.