What does Model Drift mean?

Model drift describes the loss of accuracy a model experiences in production, as reality gradually moves away from the training data. The model itself stays unchanged, but its environment does not. Unlike a software bug, drift creeps in gradually and, without ongoing measurement, is often only noticed months later, usually through complaints from a business department.

Two forms are distinguished. Data drift describes a changed distribution of inputs, for instance after a regional reorganization or the introduction of a new form. Concept drift means a changed relationship between input and outcome, for instance after a change in the law. Both can be detected by comparing the current input distribution with the one used in training, statistically for example with the Kolmogorov-Smirnov test, together with the error rate on outcomes that become known afterward.

The question of drift arises for every model that has been in use for more than a few months. A model ages especially fast in areas with frequent rule changes, such as funding programs, pricing, responsibilities, or reporting obligations. For physical relationships, such as the wear of a component, the learned relationship tends to stay stable for years.

The advantage of a fixed metric over occasional spot checks lies in when the problem is discovered. Anyone evaluating the error rate every month will notice an increase before the first complaints arrive. The nature of the deviation also indicates whether retraining with new data is enough or whether the method itself is outdated.

Drift monitoring only works with three things in place: a metric, a threshold, and a named person who acts when it is exceeded. If any of these is missing, a model keeps running unnoticed, even once its predictions are far off and decisions still rest on them.