Algorithmic Bias in Predictive Justice: How the Defence Challenges a Decision Based on a Biased Model

Predictive justice promises efficiency: deciding where to patrol, who to look at first, or what recidivism risk a person presents on the basis of statistical models. But that promise conceals an underlying risk, algorithmic bias. When a model is trained on the historical complaint and policing data of a system that already treated certain groups unequally, it does not correct that inequality: it inherits it, amplifies it and dresses it in apparent technical objectivity. As criminal defence lawyers specialising in predictive-model bias, we explain why a biased model cannot ground either suspicion or conviction, and how it is challenged by the defence.

What Predictive Justice Is and Why It Matters

Very different instruments coexist under the common label, and their impact on guarantees differs:

• Territorial models (hotspots): predict where a crime is most likely to occur and guide patrolling, with a risk of discrimination by area and of self-fulfilling prophecy.
• Individual recidivism models: score a specific person and have been proposed for decisions as sensitive as pretrial detention or parole.
• Investigative prioritisation systems: decide who to look at first, shaping the allocation of resources and the very focus of the investigation.

In all of them the problem is common: the prediction does not establish a fact, it orients the suspicion. If that orientation is biased, it contaminates the decision to investigate, the measures adopted afterwards and, ultimately, the assessment of the case.

Where the Bias Comes From

Algorithmic bias almost never arises from a discriminatory intention. It is, above all, the statistical inheritance of the data feeding the model. Four well-documented mechanisms are worth bearing in mind:

1. Reproduction of historical patterns: the model learns from past complaints and arrests and projects into the future the imbalances that already existed toward minorities or socioeconomically vulnerable areas.
2. Feedback loop: more patrolling in an area generates more complaints there, which the model reads as more crime, which justifies more patrolling. The prediction confirms itself.
3. Complaint data, not real crime: the system measures what is reported and pursued, not what truly happens, so it over-represents what was already more heavily policed.
4. Explanatory opacity: when the model is a proprietary black box, neither the person affected nor the court can verify how the score was reached.

The Constitutional Guarantees at Stake

The use of a biased model is not merely a technical problem: it touches the core of several fundamental rights. The defence articulates them together.

The presumption of innocence (Article 24 of the Constitution) prevents the investigated person's starting position from depending on the score an algorithm assigns them. It cannot operate asymmetrically: no one should bear a suspicion reinforced by their predictive profile. Equality and the prohibition of discrimination (Article 14) are directly compromised where the tool treats a person worse for belonging to a group over-represented in the data, even if the system's stated purpose was neutral.

And, centrally, the reasoning of judicial decisions (derived from the right to effective judicial protection, Article 24). A decision that restricts rights —a precautionary measure, a search, an interception— must be explained with comprehensible and reviewable reasons. If the decision rests, even in part, on the output of a model whose operation cannot be explained or reviewed, the reasoning is only apparent: it defers to a technical authority that the court itself does not control. A decision that cannot be explained cannot be reviewed, and a decision that cannot be reviewed can hardly respect the right of defence.

The European Artificial Intelligence Regulation

Regulation (EU) 2024/1689 (the AI Act) provides a useful frame of reference, even though it is not a criminal statute and does not directly regulate evidence. For present purposes it distinguishes two planes:

• Prohibited practices: among them, assessing or predicting the risk that a person will commit a crime based solely on profiling or on their personality traits. This is precisely the most dangerous scenario of individual recidivism profiling.
• High-risk systems: several uses of AI in law enforcement and the administration of justice are classified as high-risk and become subject to reinforced obligations of data governance, technical documentation, traceability, meaningful human oversight and record-keeping.

The practical consequence for the defence is clear: a system that fails to meet those guarantees —one that does not document its data, that allows no real human oversight, or that operates within a prohibited practice— is markedly easier to challenge as the basis of a criminal decision. The characterisation, in any event, is fact-specific.

How the Defence Challenges a Biased Decision

The challenge is built early and in an orderly way, and operates on two planes: the ordinary procedural one and, where the case allows, the constitutional one.

1. Auditing the origin of the suspicion: reconstructing what information or algorithm determined the initial decision to investigate the client, and asking the court for the full documentation of the model (training data, version, methodology and validation metrics).
2. Technical-statistical expert report: producing or requesting a report that documents the error rate differentiated by profile and assesses the model's bias in relation to the specific case, ideally by an independent court expert who shields the defence against proprietary opacity.
3. Attacking the indication and proportionality: if the reasonable indication that opened the proceedings was the product of an established bias, its validity is challenged, and so is that of the restrictive measures adopted on its basis.
4. Seeking nullity: where the origin of the evidence or of the measure is tainted, the nullity of proceedings may be argued and, by connection of unlawfulness, that of the derivative measures, under Article 11.1 of the Organic Law of the Judiciary (LOPJ). If total exclusion does not succeed, the reduction of probative weight is worked.
5. Constitutional route: once the judicial avenue is exhausted, the appeal for protection (amparo) before the Constitutional Court allows the violation of equality, the presumption of innocence or effective judicial protection to be denounced where the algorithmic practice has compromised them.

In parallel, where the abnormal functioning of the Administration —including the use of biased AI— has caused a provable harm, a claim for the State's patrimonial liability may be considered (Article 121 of the Constitution and Law 40/2015). The aim is not to demonise the technology, but to subject it to the same requirements of reliability, adversarial challenge and review as any other element of the process. The strategy calls for technical analysis from the outset of the proceedings and is decided case by case.