Model deployment is everything between “the model works in an experiment” and “the product depends on it”. A trained model is just a file of weights; deployment turns it into a service with inputs, outputs, owners, and a plan for when it goes wrong.
Paleyes and colleagues describe an ML project as four stages: data management, model learning, model verification, and model deployment. They define deployment as integrating the trained model into the software infrastructure that runs it, and include maintenance and updates in the same stage. Their survey of real case studies found practitioners hitting problems at every stage, but deployment is where the model first meets production constraints.
What it involves
- Choosing a shape. Online prediction behind an API (model serving), scheduled batch processing, or running on the device itself (edge deployment).
- Packaging and versioning. Pinning the exact weights, preprocessing code and dependencies, usually via a model registry, so what runs in production matches what was tested.
- Releasing safely. Rolling a new model out gradually, such as to a small slice of traffic first, and keeping the old one ready as a fallback.
- Watching it. Model monitoring tracks latency, errors and prediction quality once real data arrives.
Why it is hard
Sculley and colleagues pointed out that the ML code is only a small fraction of a real-world ML system; the surrounding infrastructure is vast. Much of the difficulty is data: a model can degrade without any code changing because the world it sees has changed. That is why deployment is treated as a continuous loop, the core concern of MLOps, rather than a one-time handoff.