Distribution shift is the gap between the world a model was trained on and the world it is used in. Standard evaluation assumes the test set comes from the same distribution as the training data. In deployment that assumption breaks all the time: a medical model moves to a new hospital with different scanners, a fraud model meets a new scam, a language model is asked about events after its training cut-off. The model doesn’t know anything changed. It just gets worse, often while staying confident.
Kinds of shift
- Covariate shift: the inputs change but the relationship between input and answer does not. Photos from a new camera, text in a new dialect.
- Label shift: the mix of answers changes but each class looks the same. Lipton and colleagues’ example is medical: disease prevalence changes while symptoms per disease stay put.
- Concept shift: the relationship itself changes. What counted as spam last year may not this year.
When the shift is gradual and ongoing in production it is usually called data drift; when a single input falls far outside anything seen in training, it is out-of-distribution.
How big a problem
WILDS, a benchmark built from real deployments such as tumour images across hospitals and wildlife photos across camera traps, found that standard training performs substantially worse out-of-distribution than in-distribution on every dataset, and that methods designed for shift did not close the gap.
What helps
There is no general cure, but there are habits. Evaluate on data from the conditions you expect to deploy in, not just a random split. Monitor inputs and outcomes after launch, and plan for retraining. Training on more varied data can improve robustness, and when you know which kind of shift you face, targeted corrections exist: Lipton and colleagues estimate new label proportions without any new labels.