A common starting point is to pick the best model you can afford and use it for every step. It works in a demo. At volume it is slow and expensive, and it hides a more useful question: what does each step actually need?
A workflow has many small jobs. Some are easy (is this email a complaint?), some are hard (reason about a contract clause), and some are risky (draft a message to a customer). They do not need the same model.
Match the step to the job#
Think of three things for each model step.
Difficulty. Classification, extraction into a fixed format, and short rewrites are well within reach of small, fast models. Multi-step reasoning, long documents and ambiguous judgment need a stronger one.
Risk. What happens if the output is wrong? A mislabelled ticket gets re-routed. A wrong figure in an approval summary might be signed off. Use the stronger model, and a person, where mistakes are costly.
Volume and latency. A step that runs on every incoming message adds up quickly. A step that runs once per approved case does not. A customer waiting on a reply feels latency; a nightly report does not.
The router pattern#
Anthropic's guide on building agents lists routing as a core workflow pattern: classify an input and send it to a specialised follow-up task. It notes that routing makes sense for complex tasks with distinct categories, and one of its examples is sending easy questions to smaller models and hard ones to more capable models to optimise cost and speed.
Here is how that looks in practice:
- A small, inexpensive model reads the input and returns a category and a rough difficulty (simple, normal, hard).
- Simple items take a fast path with a small model, or none at all if rules can handle them.
- Normal items go to a mid-tier model.
- Hard or high-risk items go to your strongest model, and often to a person afterwards.
The router must be right more often than it is wrong, and it costs something itself. Test it on real data: sample a few hundred inputs, have a person label the difficulty, and see how often the router disagrees. If the router is unreliable, route on rules instead (length, category, customer tier, presence of an attachment).
Cheap first, escalate on doubt#
A related pattern: try the small model, check the result with something that is not the same model, and escalate only when the check fails.
Checks that work without AI: does the output match the schema, are required fields present, do the numbers add up, is the category one of the allowed ones. Checks that use AI: a second step that asks whether each claim is supported by the source text. If a check fails, run the stronger model on that item.
This can cut spend a lot on workloads with many easy cases, but the saving depends entirely on your mix. Measure your own traces rather than assuming a ratio.
Plan for failure: fallbacks#
Models get rate limited, have outages and are retired. A workflow with a single model hard-wired into every step will break when that one does.
Some model gateways support fallbacks. OpenRouter, for instance, lets you list models in priority order and tries the next when the first errors, with triggers such as context length errors, moderation flags, rate limiting and downtime. You are charged at the rate of the model that handled the request. That helps availability, but remember what it changes: a fallback model may behave differently from your primary. Test the fallback on the same test set, and avoid falling back from a strong model to a much weaker one on a high-risk step without a person in the loop.
In Cadrix you bring your own model key, so you decide which provider each step uses, and a gateway is one option.
Where your data goes#
Each model call sends the step's content to the provider you choose. That has consequences for privacy and residency. Decide which steps may send personal or confidential data to a hosted model, and which should use a model you host yourself or an internal endpoint. Cadrix Enterprise, running in your own environment, lets you point workflows at an internal model endpoint, which is useful in closed networks. Check the provider's data handling terms for the hosted ones.
Track cost per workflow, not per call#
A single call price tells you little. What matters is cost per completed item, including retries and loops. Agents that call tools in a loop use more tokens than fixed workflows, and Anthropic's guide notes agents mean higher costs and the potential for compounding errors.
Use your usage data to answer:
- Which workflow consumes the most tokens?
- Which step within it?
- What is the cost per run, and per approved outcome?
Then change one thing at a time: shorten the context, move a step to a smaller model, or cache a lookup, and compare the cost and the edit rate. Quality has to stay level, or you have not saved anything.
Cadrix lists each run with its tokens and cost, and the usage dashboard shows daily tokens and estimated spend, with a monthly token cap you can set.
A simple selection table#
| Step | Typical choice | Why |
|---|---|---|
| Classify an email | Small, fast model | Easy, high volume |
| Extract fields into a schema | Small or mid model, schema validation | Format check catches errors |
| Summarise a long document | Mid model with large context | Needs context, not deep reasoning |
| Draft a customer reply | Mid or strong model, plus human review | Tone and risk |
| Reason over a policy exception | Strong model, plus human | Hard and consequential |
| Judge a draft against sources | Different model from the drafter | Reduces shared blind spots |
These are starting points, not benchmarks. Model lineups change often, so test the current options on your own data and write down the date you did.
Building it in Cadrix#
- A Start step feeds a small LLM step that classifies the input and rates difficulty.
- A Condition routes by category and difficulty to different branches.
- Each branch uses an LLM or Agent step configured with a different model connection.
- A validation Condition checks schema and arithmetic; failures loop to the stronger model branch.
- High-risk branches end in a Human Input step before any outward action.
- Check the run list for cost per run, and compare branches over a week.
