By Karl Lehnert, Director, DevProStudio
Published 19 August 2026 · Last reviewed 19 August 2026
The “best model” is rarely best at everything. A model that handles a messy contract well may be needlessly expensive for inbox classification. A fast model may stumble on a tool-heavy support request. And the strongest benchmark model may be off-limits for personal information.
That gap is a live 2026 engineering concern. On 5 August 2026, Latent Space linked rising routing demand to frontier-model cost and open-weight adoption. For an Australian accounting, legal or NDIS services firm, the same inbox can contain routine admin, complex documents and sensitive personal information. Sending every item down one model path is a cost decision and a data-handling decision.
What model routing actually means
Model routing is a decision layer placed between an application and the models it can call. It examines a request, applies policy, chooses an eligible model, and records what happened. A robust router can also escalate when confidence is low or a validation check fails.
The working sequence is:
- Classify the task and its risk.
- Remove any models that are not allowed for that data or tool access.
- Choose from the remaining models using quality, cost and latency evidence.
- Validate the result.
- Escalate, retry or send to a person when necessary.
Privacy and permission rules come first. A cheap or capable model is irrelevant if it is not approved for the prompt data.
AI model routing cost in Australia
Vendor price tables make comparison look simple. OpenAI, Anthropic and Google publish current API rates on their official OpenAI, Anthropic and Gemini pricing pages. Those rates vary by model and can also treat cached input differently.
But token price is only the visible part of the bill. The useful metric is cost per accepted outcome:
(model calls + retries + tool charges + review time) ÷ accepted outputs
Suppose an inexpensive route regularly needs a second call and five minutes of staff correction. Its headline token rate may still look attractive while the completed task costs more. The reverse also happens: using a frontier model for routine extraction can produce an excellent answer that the business never needed to pay for.
Latency belongs in the calculation too. A background document job can wait. A customer-facing interaction cannot. Route design should reflect the business process, not a single leaderboard.
Start with two routes, not ten
The common mistake is to connect every interesting model, then invent rules for all of them. That creates a brittle model zoo. Each route adds evaluation work, provider risk, observability requirements and another failure mode.
A more useful first version has two paths:
- Default route: economical, fast and proven on routine tasks such as classification, structured extraction or first-pass drafting.
- Escalation route: reserved for complex reasoning, higher-value outputs or cases that fail a validation check.
Keep a human-review path for sensitive or ambiguous work. Add a third model only when your evaluation set shows a stable class of requests that it handles materially better.
Models change quickly. Routing policy should describe business requirements—data class, latency, output schema and acceptance threshold—not marketing names.
Multi-model LLM routing scorecard for SMEs
Before writing code, build a small evaluation set from representative, de-identified tasks. Include easy cases, edge cases and failures that matter. Record:
- task acceptance rate;
- cost per accepted result;
- response time, schema validity and tool success;
- review minutes, fallbacks and retries; and
- provider, location and retention constraints.
Do not hide this in one magic score. A lower price cannot compensate for a privacy violation or invalid financial calculation. Apply hard gates, then rank eligible options.
The NIST Generative AI Profile frames generative AI as something to measure and monitor. A spreadsheet and versioned test set can be enough.
AI document triage for an Australian services firm
Consider “Harbour Advisory”, an explicitly fictional 12-person Sydney accounting firm—not a DevProStudio client or claimed case study. Its emails and attachments must be classified, summarised and assigned.
The router first identifies whether the item may contain personal or commercially sensitive information. Policy determines which providers are eligible. A lower-cost model handles straightforward classification into a fixed schema. If required fields are missing, confidence is below the agreed threshold, or the document is unusually long or complex, the job moves to the escalation model. High-risk categories always require staff approval before any downstream action.
The application records the route, model version, validation result, elapsed time and approval status. It avoids copying the full sensitive document into operational logs. Over time, the team reviews false classifications and adds them to the evaluation set.
This pattern does not let an agent decide everything for itself. Routing should expose uncertainty, not bury it behind another model call.
Privacy Act controls for AI model routing
Model routing can improve data control, but only if policy is built into the route. The Australian Privacy Principles include requirements relevant to use and disclosure of personal information (APP 6) and reasonable steps to protect it (APP 11). The OAIC also publishes specific guidance on commercially available AI products.
Start by classifying data, minimising prompts and redacting identifiers. Maintain a provider-and-model allow-list for each data class. Credentials and tools need their own restrictions; model selection is not access control. Document retention, region and contract settings. Log decisions without copying sensitive content, and test a manual fallback.
Sending the same payload to several models “to see which answer wins” increases exposure. In production, parallel calls should be a deliberate, justified exception.
Cost of AI model routing: code or managed gateway?
Application-level routing has no separate software licence when you write and host it; engineering, testing and monitoring remain real costs. Open-source gateways may remove a licence fee but not operations. Managed gateways commonly use usage-based or negotiated plans, so obtain a written quote.
For a concrete planning scenario—not a vendor quote—assume 10,000 monthly jobs. If routing saves two cents per accepted job after retries and review, the gross saving is $200 a month. If it saves 20 cents, it is $2,000. Compare that with gateway fees and maintenance. Replace both assumptions with a pilot’s measured figures and current rates from the linked official price pages before approval.
The sensible 30-day approach
Pick one repeatable, measurable workflow. Assemble 30–50 de-identified examples from the real task, define hard policy gates, and compare a default model with one escalation model. Run in shadow mode before allowing automated downstream actions. Review misses weekly and version the test set.
After a month, you should be able to explain why each route exists, what it costs per accepted result, where it fails, and which data it may handle. If you cannot, adding more models will add complexity rather than value.
Talk to an AI routing engineer
DevProStudio designs practical custom AI applications for Australian businesses, including their evaluation, integration and controls. If your AI costs are climbing or one model is stretched across incompatible jobs, talk to DevProStudio about a measured routing pilot.
Related reading: Open Weights or Closed APIs and Stop Buying AI Tools Without Measuring Unit Cost.
Frequently asked questions
What is AI model routing?
AI model routing is a software layer that selects an eligible model for each request using task type, data policy, expected quality, cost and latency. Strong implementations also validate results and provide escalation or human-review paths.
Does model routing reduce AI costs?
It can, when routine work moves to an economical model without increasing retries or correction. Measure cost per accepted outcome rather than token price alone, including tool charges, review time, failures and fallback calls.
How many models should an SME start with?
Two is usually enough: a proven default model and a stronger escalation model. Add another only when a representative evaluation set demonstrates a stable task segment with a meaningful quality, cost, latency or data-control advantage.
Is model routing safe for personal information?
Routing is not automatically safe. Classify and minimise data first, restrict eligible providers and regions, apply APP obligations, control tool credentials, avoid sensitive prompt logs and require human approval for high-risk actions.