The Pilot Worked Because the Exceptions Were Still Manual

The pilot met its target. Turnaround improved. Users were satisfied. The rollout was approved.

What the summary did not show was the project manager correcting records at night, the specialist answering every edge case in chat, and the operations team manually moving work whenever the new process stalled.

The pilot worked. The design may not have.

Pilots are supposed to expose problems before scale. GAO’s Agile Assessment Guide describes a pilot as a way to discover problems, mature processes, and incorporate lessons before broader adoption. GAO has also warned that programs risk scaling without understanding the factors that produced the pilot’s results when evaluation requirements are weak.

Manual support is not automatically a flaw. Early learning often requires close observation and hands-on recovery. The mistake is counting that support as if it were part of a repeatable system.

Create an exception log during the pilot. For each intervention, record what triggered it, who resolved it, how long it took, what access or judgment was required, and whether the scaled process will contain that capability. Group the exceptions rather than dismissing them as isolated events.

Pay special attention to work performed by unusually experienced people. A pilot staffed by designers, senior operators, and executives can move quickly because knowledge and authority are concentrated nearby. A scaled operation may rely on new employees, shared support, standard permissions, and slower escalation.

Test volume and variation, not just the happy path. Increase the workload. Introduce incomplete inputs, late requests, unusual customers, system downtime, and staff absence. Observe recovery. A process that succeeds only when experts are watching is still a prototype.

The scale decision should include the hidden operating model: expected exception rate, support capacity, training load, monitoring, fallback procedure, and cost of manual intervention. If those items are unknown, the business case is incomplete.

Some workarounds should become designed controls. Others signal that the process, policy, or technology needs to change. A few may remain manual because automation would cost more than the problem. The important thing is to choose deliberately.

Pilot success is not a ceremony. It is evidence. Preserve the conditions that matter, remove the conditions that cannot scale, and retest before the spotlight moves on.