Why Health-Technology Pilots Succeed and Then Fail to Scale

A health-technology pilot succeeds and then fails to scale because it quietly supplied everything the product could not yet supply on its own. Eager early users, a hands-on team, a forgiving site, attention from leadership: those props carry a tool past problems it has not actually solved. Scaling removes them, and what is left has to stand by itself in a clinic that did not volunteer, on a week when no one from the company is in the building. The gap is the distance between a result the conditions produced and one the product produces.

I have lived on both sides of this. I co-developed an AI decision-support system for type 2 diabetes and ran it through a randomized trial across more than forty clinics, and I later co-founded a company around an AI symptom checker that was acquired. I have also watched tools that looked unstoppable in a pilot vanish a year into rollout. The pattern is consistent enough that I now distrust a clean pilot, since a clean one usually means the conditions did the work.

What is the pilot-to-scale gap?

The pilot-to-scale gap is the difference between how a tool performs under the special conditions of a pilot and how it performs once those conditions are gone. A pilot measures the product plus its support. Deployment measures the product alone.

A scalable product is one whose value survives the removal of everything that made the pilot easy: the volunteer site, the embedded team, the curated patients, the patience of leadership, the novelty. A pilot tells you the ceiling under ideal conditions and almost nothing about the floor. An improvement often reflects the product plus a hard-working implementation team, and that team does not ship in the box.

Why does a successful pilot mislead you?

A pilot misleads because almost every variable that makes it succeed is one you cannot ship. Start with who runs it. The first sites self-select for enthusiasm. They have a champion, a clinician who wants the thing to work and will route around its rough edges rather than abandon it. That person is real and valuable, but not representative. The average deployment site has no champion, only a busy team told a new tool is coming. A product that needs a believer to function has not been validated. It has been carried.

Attention inflates the result too. During a pilot, the people who built the tool watch closely, fixing problems within hours and answering questions in a shared channel, so you end up measuring the product with its creators standing behind it. At scale they are spread across hundreds of sites and the response time is measured in weeks, if it exists at all. Novelty adds its own lift, and that lift fades. What remains is whatever value the tool delivers once it is ordinary, which is the only value that ever scales.

What actually breaks when you go from ten clinics to a thousand?

What breaks first is the part the pilot held together by hand, and the hand does not scale. In a pilot you can clean the inputs and fix the one site whose records are a mess. A thousand sites bring a thousand dialects of the same record: missing values where you expected numbers, units that differ, habits no one wrote down. A model that performed well on tidy pilot data can sag on the real distribution. I learned this early in diabetes research, where a relationship that looks fixed in one population can change shape in another.

Workflow is next to give. The pilot site had one way of working that you fit the tool into carefully. The next thousand each have their own: the order things happen in, who touches the chart, where the decision gets made. A tool that assumed the pilot's choreography becomes friction wherever the choreography differs. Fit is not something you establish once. You earn it against how real clinics run.

Most underestimated is the support model. Whatever the pilot solved by having a human on call has to be solved by the product itself, or by a support operation you have not built and may not be able to afford. A tool that leans on its makers answering questions has outsourced part of itself.

How do you build a pilot that predicts scale?

You design the pilot to remove its own props, deliberately, before reality removes them for you. Test at least one site that did not ask for the tool and has no champion, since the unmotivated site is the honest preview of deployment. Let the product face records as they actually arrive, and pull your own team back for a defined stretch to see what happens when help is days away. All of it is cheaper to learn in a pilot than in a failed rollout.

Define success the way a trial does, against a claim that could have failed. When my colleagues and I tested EASY Diabetes, the study was built so it could have come out the other way, across more than forty clinics, against standard care. A test that takes a real risk and survives it says what a flattering pilot never can.

What does crossing the gap actually take?

Crossing the gap is its own discipline, separate from building the thing, and it gets a fraction of the respect. It means engineering the product so the easy parts of the pilot are handled by software rather than people, earning fit against many workflows, and planning for life after launch, since performance drifts as populations and practice patterns move. It also means honesty about the regulatory claim, since what you are allowed to say shapes the evidence you must gather. I took training in medical device regulations at KTH because that work is not optional once a tool shapes a clinical decision.

A pilot and a deployment are almost different products that share a codebase: one is a demonstration under supervision, the other has to survive without it. Teams that scale treat the pilot as a way to find what the product cannot yet do alone, then close each gap before the props are gone.

This article is educational and is not medical advice. For decisions about your own care, talk with your clinician.