Plenty of AI automation projects start with enthusiasm and end quietly: a demo that impressed, a pilot that never reached production, a budget line that disappeared. It helps to know why before you start yours.
What the forecast says#
In June 2025, Gartner published a press release titled "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027". Reporting on the release attributes the forecast to escalating costs, unclear business value or inadequate risk controls. Gartner also warned about "agent washing", where vendors relabel existing assistants, chatbots and RPA tools as agentic, and estimated that only about 130 of the thousands of vendors claiming agentic capabilities are genuine.
Two cautions about this number. It is a prediction, not a measurement, and it is more than a year old as we write. Treat it as a reason to plan carefully, not as a fact about your project. The causes it names are still the useful part, because they match what teams run into.
Five reasons pilots stall#
1. The use case is a showcase, not a job. The pilot automates something impressive but rare, so nobody depends on it. When the sponsor's attention moves, it dies. Choose work that happens daily and that someone is tired of doing.
2. No baseline. Without knowing how long the task takes today, how often it goes wrong and who does it, you cannot show improvement. "It feels faster" does not survive a budget review.
3. Cost surprises. Model calls, retries and long contexts add up, and an agent that loops uses more than a fixed workflow. A pilot at ten runs a day tells you little about a thousand.
4. Risk is discovered late. Someone from security or legal sees the project when it is nearly done and asks questions it cannot answer: what data does it see, who approved that action, where are the logs? Bring them in at the start.
5. Nobody owns it after launch. Prompts drift out of date, a source system changes a field, a model is retired. A workflow with no owner decays.
A 30-day pilot#
This plan assumes one workflow, one team, and a clear owner. Adjust the pace to your organisation.
Week 0: Choose and measure (before the clock starts)#
Pick one job that meets these tests:
- It happens at least several times a week.
- The input is mostly consistent.
- A person can check the result in a minute.
- A mistake is recoverable.
- The people who do it today are willing to help.
Then measure the current state for a representative sample: time per item, error rate, volume, and who touches it. Write the numbers down. Examples that fit well are invoice intake, support triage, onboarding document checks and internal request routing.
Week 1: Build the thin version#
Build the smallest workflow that does the whole job end to end, with a human approval at every consequential step. Do not polish. Connect only the systems you need, and use sample or copied data first. Write down which data the workflow reads and which actions it can take; that list is the start of your risk review.
In Cadrix this means drawing the workflow on the canvas, attaching only the tools it needs, adding an approval step before any action that changes something outside, and testing runs from the editor with the trace open.
Week 2: Shadow mode#
Run the workflow on real incoming items but do not act on its output. Compare what it would have done with what people actually did. Count agreements, disagreements and cases it could not handle. Fix the top causes of disagreement: usually a missing rule, a vague instruction or missing knowledge.
Week 3: Supervised live#
Switch on live actions with approvals. Every outcome passes through a person who can approve, edit or reject. Track the edit rate and the rejection reasons. Record the cost per run from the usage figures. Hold a short weekly review with the people using it.
Week 4: Decide#
Reduce approvals where the evidence supports it: for categories with a very low edit rate and low stakes, allow straight-through processing with a sampled review. Keep approvals on anything costly or irreversible. Then use the scorecard to decide whether to expand, fix or stop.
The scorecard#
Score each line from the evidence in your traces and your baseline, not from impressions.
| Measure | How to get it | Decision signal |
|---|---|---|
| Time per item vs baseline | Run durations and human review time | Is the total lower, including review? |
| Edit and rejection rate | Approval records | Falling over the weeks? |
| Error rate that reached a customer or system | Incident log | Zero for high-impact errors? |
| Cost per run | Usage and cost per run | Below the value of the time saved? |
| Coverage | Share of items handled without escalation | Enough to matter? |
| Risk review | Data list, action list, access, logs | Signed off by security or legal? |
| Owner and runbook | Named person, written steps for failures | Exists? |
| User sentiment | Short survey of the team | Would they miss it? |
Agree the thresholds before the pilot, not after. If you set them after seeing results, you will choose numbers that flatter the result.
Building the habits that prevent stalling#
- Name an owner for each workflow, with a backup.
- Review failed and rejected runs weekly during the pilot, monthly afterwards.
- Keep a changelog of prompt, model and tool changes, and re-run your test set after each one.
- Set a spending cap or alert so cost surprises arrive early. Cadrix tracks tokens and estimated spend and lets you set a monthly token cap.
- Treat the pilot as a test of the process, not only the technology.
When to stop#
Stopping is a valid outcome. If the edit rate stays high, the cost per run exceeds the value, or the risk review cannot be signed off, the pilot has done its job by telling you cheaply. Write down what you learned, and use it for the next candidate.
