routing
Picking the right model per request beat always using the biggest one News
A new routing framework that chooses a different model for each request outperformed the strongest single fixed model by 14.6 percent, partly because the largest model gets many cheap questions wrong.
Model Routing and Cascades: Spending Frontier Money Only Where It Buys Something Lesson
Model routing sends each request to the cheapest model that can handle it, and cascades escalate to a stronger model only when a cheaper one is not confident, which cuts serving costs substantially because most production traffic is easy.
A Show HN Promised Frontier Quality for Half the Cost. Its Repo Describes a Router. News
World Model Optimizer turns agent traces into a policy that sends easy requests to cheap models and hard ones to a frontier endpoint, but its headline savings claim shifted between the repository and the post, and no distilled model has been released.
World Model Optimizer Tool
A pip-installable CLI that turns the OpenTelemetry traces your agents already emit into a routing policy: it scores every model you have registered against held-out tasks from your own traffic, then serves an endpoint that sends easy requests to cheap models. Treat the routing as the product; the distillation half has no released checkpoint yet.
Sakana Fugu Tool
A single OpenAI-compatible endpoint that dynamically routes each request across several frontier models, so you call one API and get a coordinated multi-model answer.
OpenRouter Tool
A production gateway to hundreds of models behind one API, with public rankings built from real usage and the ability to sort by price, throughput, latency and popularity.
NeMo Switchyard Tool
NVIDIA's library for routing each task in a multi-model system to the model best suited to it, so a frontier model handles planning while a cheaper one handles execution. Shipped alongside Nemotron 3.5 Lightning as the connective tissue for mixed-model agent stacks.
LLMRouter Tool
A unified framework for building, evaluating and deploying model routers, with a quickstart, single and batch routing calls, and a benchmark that dispatches queries across eighteen candidate models with cost tracking.
Cactus Hybrid (Gemma-4 E2B) Tool
A phone-sized Gemma-4 checkpoint with an attached error probe that scores how likely each answer is wrong and routes low-confidence queries to a cloud model; weights and runtime are public (set CACTUS_CLOUD_STRICT_SSL before using the cloud path).