Where Good Pilots Go to Die
By Jim Rubino - Head of Delivery
11/04/2026
A pilot finishes on a good note. The small team that ran it doesn't want to give the tool back. The sponsor wants it in every department, and the results hold up when finance goes through them a second time. Money for a wider rollout comes through without much of a fight.
Six months later the same tool is installed across the company, and the teams using it are working through the same steps they always did. Some people open it daily. The gains the pilot showed haven't come anywhere close to what people expected, and the explanations on offer don't quite fit.
The pilot result was real
After a rollout stalls, it's tempting to go back and doubt the pilot. Perhaps the team was hand-picked, or the use case was easier than the real thing. I'd resist that explanation. In a pilot worth the name, the tool did what it was tested to do. The people running it were capable and committed, and nobody arranged the outcome to look better than it was.
Doubting the pilot is also the cheapest explanation available. If the result was a fluke, nothing else needs examining. If the result was real and the rollout still stalled, then something that was present in the pilot didn't travel with the tool, and that's worth finding.
The pilot team didn't have rarer talent than the teams that came after them. They had different conditions.
The pilot changed the work, and nobody called it that
Set the tool aside for a moment and look at what else happened inside the pilot. Say the team used to rebuild a tracking spreadsheet by hand every Friday. Partway through, someone decided the tool's output covered it, and the spreadsheet stopped. No one changed a policy. The step lapsed because the pilot was allowed to let it lapse.
Other things moved too. If the tool was blocked by a permission it lacked or a system it couldn't reach, the sponsor got it cleared within a day, because the pilot had the sponsor's attention. The people in it had volunteered, and they had protected time to try things and say what wasn't working. Each of these is a change to how the work runs. Taken together, they're a redesign of a small team's work.
The technical side has its own version. Pilot access is usually granted by exception: someone in IT approves wider permissions for a small group because the arrangement is temporary. The test data gets picked because it's clean enough to show the tool working. Where two systems don't connect, whoever's nearby bridges them with an export or a short script. None of that is wrong for a test. It does mean the pilot runs in a hand-arranged environment, and the rollout meets the standard one, with ordinary permissions and the data as it actually sits.
Nobody filed any of this under results. The lapsed steps and the fast answers were treated as setup, the scaffolding you put up so the test can happen and take down afterward, so they never show up in what the pilot is said to have proven. They usually weren't written down either.
A pilot is a small, temporary redesign of the work, and the organization remembers it as a test of the tool.
A pilot is a small, temporary redesign of the work, and the organization remembers it as a test of the tool.
The rollout ships what can be scheduled
A rollout plan is built from things that have an owner and a date. Licenses get bought, access gets provisioned, training sessions go on calendars and the go-live gets a specific Monday. Somebody is accountable for each of those, which is why they happen.
The pilot's changed conditions had no owner and no date. Nobody was responsible for the Friday spreadsheet stopping. It stopped because a sponsor was paying attention and a team had room to try. So those changes stay behind with the pilot team, and the plan carries the tool out without them.
Take the team lead on the receiving end. That team gets logins and an hour of training. They don't get permission to drop the Friday spreadsheet, so they still build it, and now someone also checks the tool's output against it. If the tool can't reach a system, the request joins the normal IT queue and waits its turn. The approvals that applied last year still apply. The people on that team are doing what their week asks of them, and their week didn't change, so the tool arrives as one more thing to do. Once it's an extra step, use tends to settle into small, safe requests, the pattern described in The Vending Machine Problem.
The rollout has to carry the changed work
I'd read a successful pilot as evidence about the work, and only partly as evidence about the tool. It shows what that work looks like when a few of its rules are lifted. A rollout that holds has to carry the pilot's decisions about the work with it: which steps stop, and who answers within a day when the tool is blocked. Sometimes it also has to change what a team is accountable for, because the team now owns different output than it did before.
Those decisions sit above the pilot team. A pilot team can let its own spreadsheet lapse. It can't stop another department's approval step or retire a report a senior leader still reads. Carrying the changes means reaching into work that belongs to other people, outside anything the pilot was scoped to touch, and that's uncomfortable enough that it usually gets skipped. The tool is easier to ship.
For whoever funds the rollout, this changes what a stall means. The pilot result was real and the receiving teams are capable, so neither the tool nor the people explain it. Funding a wider rollout of the same tool on the same plan buys a larger stall.
The pilot team ends up on the standard rollout like everyone else. Their sponsor moves on to the next priority, and the access granted for the test gets tidied up in the next audit. The Friday spreadsheet comes back because someone downstream asks where it went. The redesign they ran ends the way it started, without anyone calling it one.
Head of Delivery, Arkane Digital
Jim focuses on executing complex AI and digital initiatives, ensuring strategy translates into delivery across teams, systems, and timelines.