AIYour Data Science Team Isn't Shipping. Here's Why.
I’ve now had some version of this conversation about a dozen times.
A company hired data scientists two years ago. The people are good. The models genuinely work, in the sense that the validation numbers are real and the approach is sound. And almost nothing has made it into a production system that affects a customer or a decision.
Everyone involved is frustrated and each side has a theory. The business thinks the team is academic. The team thinks the business doesn’t understand. In my experience both are wrong and the actual cause is structural.
The Notebook Is Not the Problem, But It Is the Symptom
A Jupyter notebook is an excellent research tool. It is a terrible production artefact and everyone knows this, which is why “get it out of the notebook” is such common advice.
The advice is useless on its own, because the notebook isn’t a choice. It’s what you get when there is no defined path to anywhere else.
Ask the question directly: when a model works, what specifically happens next? In most companies I’ve seen, the answer is a shrug, a discussion, and a negotiation with an engineering team who have their own roadmap and no obligation to help.
“Nobody decided models would live in notebooks forever. Nobody decided anything, which is a decision with the same outcome and no owner.” — Sameer Gupta
The Six Things That Are Missing
I’d characterise the gap as six specific pieces of infrastructure and ownership, and most organisations have none of them.
- A serving path. Somewhere a model can run on a schedule or behind an endpoint, with the same data available to it that it had in training. This sounds trivial and it is the single most common blocker.
- Feature consistency. The features in training came from a warehouse with full history. The features at inference must come from a live system, computed identically. Any drift between the two produces a model that scores well in testing and poorly in production, and the cause is very hard to find.
- Monitoring. Not uptime. Distribution of inputs, distribution of outputs, and accuracy against outcomes once they arrive. A model that has silently degraded is worse than one that’s down, because nobody knows.
- A retraining process. Models decay. Somebody has to own re-fitting on new data, validating it, and promoting it, on a schedule, forever.
- Rollback. When the new version is worse, you need to return to the previous one in minutes. That requires versioning models the way you version code, which almost nobody does.
- A named owner after launch. The data scientist moves to the next project. Who is responsible for this model in a year? If the answer is nobody, it will rot, and the first sign will be a business problem nobody connects to it.
Why It Persists
The structural reason is that these six things sit between two teams and belong to neither.
Data scientists are hired for modelling and often measured on models built. Building infrastructure is not what they were hired for, they’re frequently not the best people to do it, and it doesn’t get them promoted.
Software engineers have a roadmap of committed work. A request to operationalise a model arrives as an unplanned ask from a team they don’t report to, for a system with unfamiliar failure modes. It goes to the bottom of the backlog, sincerely.
So the work falls in the gap, and the gap has no budget line.
“You funded the research and you funded the platform. Nobody funded the distance between them, and that distance is where every one of these projects has stalled.” — Sameer Gupta
What Actually Fixes It
Fund the gap explicitly. One or two engineers whose entire job is getting models into production and keeping them there. Not a rotation, not a favour. A funded role with an owner. This single change fixes more of these situations than anything else I’ve seen.
Make deployment the definition of done. A model that isn’t serving predictions has produced no value. If your team’s metrics count models built rather than decisions changed, you’re measuring the wrong thing and you’ll get what you measure.
Start with the smallest useful thing and ship it end to end. A mediocre model in production teaches you more than an excellent one in a notebook, because it surfaces every one of the six problems above while the stakes are low. I’d rather a team ship a logistic regression in month two than a beautiful ensemble in month fourteen.
Put the data scientist on call for their model. This is unpopular and it works. Nothing improves a person’s interest in monitoring and graceful failure like being the one who gets paged.
Count the running cost from the start. Every model in production is a permanent obligation. If you have twenty models and no platform, you have twenty individually maintained systems and a growing problem.
Final Thoughts
There’s a term starting to circulate for this discipline, and I’m ambivalent about naming it because naming things tends to produce vendors selling the named thing rather than teams doing it.
But the underlying observation is right. The bottleneck in this field moved. Five years ago it was whether you could build a model at all. Now it’s whether you can run one reliably, watch it, and replace it.
That’s an engineering and organisational problem wearing a data science costume, and it will not be solved by hiring more data scientists. If your team is stuck, count how many of the six things above you actually have. The number is usually zero or one, and that tells you where the next hire should go.