Using Model Routing to Lower Inference Costs
InfoWorld, Wednesday, October 21st, 2026
Martin Heller surveys model routing, which sends each request to the cheapest model able to handle it, and the tools that offer it.
Martin Heller explains model routing, which automatically picks a model based on task complexity, the available model pool and an optimization goal such as cost or quality.
He surveys implementations including Databricks Smart Routing, Microsoft's model router with balanced, cost and quality modes, Not Diamond, NVIDIA NeMo Switchyard's classifier, stage, escalation and trainable routers, OpenRouter Auto Router, Replit and Snowflake Cortex AI Gateway.
He also cites the RouteLLM research.
Routing can cut cost and average latency substantially, but results depend on how well the router judges which tasks need a frontier model.