Research is finished when it runs

There is a moment in every machine learning project where the metrics look good and everyone relaxes. The model separates the classes. The validation curve behaves. Somebody puts a number in a slide.

That moment is roughly a third of the way through the work, and it is very often mistaken for the end.

I spend my time on both sides of this line — research on medical imaging, and consulting work getting models into systems that people use on a Tuesday. The gap between the two is not a deployment step. It is a set of things that were never part of the notebook and have to be designed in, usually earlier than anyone wants to.

The notebook lies about four things

It lies about the data. In a notebook you have a dataset. In production you have a stream of records arriving in a format defined by someone else, with missing fields, encoding problems, and a bank that changed its descriptor format last month without telling anybody. The cleaning code in the notebook was written once, against data that had already been cleaned once. That code is not the pipeline. It is a sketch of what the pipeline will have to do, forever, unattended.

It lies about latency. Nobody measures inference time in a notebook because nothing is waiting. Something is always waiting in production, and it is usually a person looking at a screen.

It lies about the decision. A notebook produces a prediction. A system has to produce an action — accept, propose, escalate, refuse. Between the prediction and the action sits a threshold, and that threshold is a business decision dressed as a hyperparameter. Set it high and the system saves little effort. Set it low and the first visible mistake destroys the operator’s trust, permanently. The number is not found in validation. It is found in use.

It lies about time. A notebook is a photograph. The world in it does not move. The business changes, the data drifts, and the model that was right in March is quietly wrong by November — and nothing about the notebook will tell you which week it happened.

What actually closes the gap

Decide where the model lives before you train it. This is the decision I would defend hardest and the one most often made last. A model that runs inside the screen an operator already uses gets used. The same model, more accurate, in a separate tool, does not — because nobody opens it. A correct answer in the wrong place is not an answer. If that constraint means choosing a library that fits the existing runtime instead of the one you would pick on merit, choose the one that fits. The constraint is the requirement.

Ship confidence, not just prediction. A prediction alone forces the system to pretend it is always right. A prediction with a confidence lets you build the only workflow anyone will accept: high confidence proposed and accepted at a glance, low confidence escalated to a human. This is also what makes the system measurable in terms the business recognises — not accuracy, but how much reviewing it removed.

Capture corrections. When someone overrides the model, that override is the most valuable data you will ever get: a labelled example of a case you got wrong, produced by an expert, for free. A system that discards it is a system that can only get worse relative to a world that keeps moving.

Build explainability in, not on. In medical work this is not negotiable — a clinician will not act on a prediction they cannot interrogate. But the constraint is healthy everywhere. It shapes the architecture: branches you can ablate, features you can point at, a preprocessing chain you can walk backwards. Retrofit it and you usually end up changing the architecture anyway, later, under worse conditions.

Make the pipeline reproducible before you make the model better. A model you cannot rebuild is a model you cannot fix. Reproducibility is not process hygiene; it is the difference between an artefact and an anecdote.

Why this is a research problem, not an engineering one

The instinct is to file all of this under engineering — the messy part after the science. That framing is what causes the problem.

Preprocessing decides what the model can possibly learn. The threshold decides what the system actually does. Explainability constrains which architectures are even admissible. These are not downstream of the research; they are inside it. Treat them as someone else’s problem and you will produce a model whose excellent metrics describe a task nobody has.

A model in a notebook has not been finished. It has been described. The work is finished when it runs — and when somebody who is not you can tell that it is still running correctly six months later.