What does Bias mean?

Bias refers to a systematic distortion in the results of an AI model that regularly favors or disadvantages certain groups or characteristics. The distortion is not a random error in individual cases but repeats in the same direction with every application. It arises in the training data, in the chosen target metric, or in the way a model’s quality is measured.

Data bias occurs when the training data does not reflect the later target population, for instance because a group is barely represented in the dataset. Label bias arises when the assigned target values themselves contain an earlier form of unequal treatment and the model carries that practice forward. Algorithmic bias stems from the chosen target metric: optimizing for short processing time disadvantages the more complex cases. None of this becomes measurable until error rates are reported separately by group.

Bias becomes a problem wherever a model classifies people or has a say in access to something scarce. This affects the pre-selection of job applications, the assessment of credit applications, the prioritization of treatments, and the allocation of funding. In purely technical applications, such as forecasting wear and tear, a distortion carries far less weight.

The benefit of evaluating results separately by group is that a distortion comes to light before those affected report it or file an objection. A documented breakdown of the training data reveals gaps before deployment. And when the target metric is chosen deliberately and justified, the discussion shifts from the technology back to the underlying business decision.

Distortions are as much a part of machine learning as measurement error is part of metrology. The EU AI Act therefore does not aim for freedom from error but for evidence. For high-risk systems, it requires a vetted data basis, error rates reported by group, and clearly assigned responsibility for the effects.