Defence Against Bias in Predictive Models in Criminal Investigation

Predictive models applied to criminal investigation —from territorial predictive policing to individual recidivism evaluation algorithms— promise efficiency but bring a structural risk: algorithmic bias. When a model is trained with historical police and judicial data, it inherits and amplifies the system's previous biases. Criminal defense against investigations whose initial basis is algorithmic requires questioning the very origin of suspicion.

Types of Predictive Models in Investigation

It is worth distinguishing the families of models, because their impact on guarantees differs. Territorial models (hotspots) predict where a crime is most likely to occur and guide patrolling, with a risk of discrimination by area. Individual models of profiling and recidivism score a specific person and have been proposed for decisions on pretrial detention or parole. Pattern detection models track operations (drug trafficking, money laundering) across large volumes of data. And investigative prioritization systems decide who to look at first. In all of them the problem is common: the prediction does not establish a fact, it orients the suspicion; and if that orientation is biased, it contaminates everything that follows.

Documented Biases and Their Origin

Numerous studies (including works on COMPAS in the US and European models) have documented: (1) reproduction of historical discriminatory patterns toward ethnic minorities and socioeconomically vulnerable areas; (2) feedback loop between algorithmic patrolling and complaint data (more police → more complaints → more prediction); (3) overfitting to complaint data and not real crime; and (4) explanatory opacity when models are proprietary black boxes. The origin of the bias is almost never a discriminatory intention: it is the statistical inheritance of a system that already treated certain groups unequally, now dressed in apparent technical objectivity.

Impact on Procedural Guarantees

When an investigation originates in a predictive output, defense can challenge: (a) the reasonable indication that justified opening proceedings, especially if the algorithm is biased against the accused's profile; (b) the proportionality of restrictive measures adopted; and (c) the presumption of innocence, which cannot operate asymmetrically according to the suspect's predictive profile. EU Regulation 2024/1689 reinforces this analysis by classifying several of these models as "high risk" and by prohibiting certain uses based exclusively on profiles.

Exclusion and Limitation Strategies

On the ordinary procedural plane, the aim is twofold: to exclude or to limit. If the reasonable indication that opened the investigation was the product of an established algorithmic bias, the nullity of proceedings may be sought and, by connection of unlawfulness, that of the derivative measures (searches, interceptions) under Article 11.1 LOPJ. When total exclusion does not succeed, the limitation of probative value is worked: a technical-statistical expert report documenting the error rate differentiated by profile lowers the weight of the evidence. To shield the defense against proprietary opacity, we ask the court to appoint an independent judicial expert to audit the model used by the prosecution.

Constitutional Action Before the TC

When the case allows, the procedural avenue is complemented by the constitutional one. The appeal for protection (amparo) is the channel to denounce the violation of fundamental rights —equality, presumption of innocence, effective judicial protection— when the algorithmic practice has compromised them, and it allows doctrine to be set on AI and criminal process. In parallel, when the abnormal functioning of the Administration (including the use of biased AI) has caused a provable harm, a claim for the State's patrimonial liability is available (Art. 121 CE and Law 40/2015). In relevant cases we reinforce the expert evidence and the argument with the collaboration of academic groups in artificial intelligence and law.

Lawfulness of algorithmic evidence: the Article 11.1 LOPJ filter

The defence's first battle is not over what the predictive model concludes, but over how the data feeding it were obtained and under what legal cover the tool was deployed. Article 11.1 of the Organic Law of the Judiciary is categorical: evidence obtained, directly or indirectly, by violating fundamental rights and freedoms shall have no effect. This exclusionary rule is not a technicality but the safeguard that prevents a prediction built on unlawful access to communications (Article 18.3 of the Constitution) or to personal data from becoming the basis of a conviction. If the source data are void, they contaminate the result derived from them.

In practice, this requires examining the judicial authorisation behind each intrusion. Interception of electronic communications requires a reasoned authorisation under Articles 588 bis a and 588 ter a and following of the Criminal Procedure Act; access to the content of mass-storage devices is governed by Article 588 sexies, which demands a specific ruling that is not implied by the mere seizure of the device. When a predictive analysis system draws on bulk extractions or database cross-referencing without cover in these rules, the defence may argue that every subsequent inference is reached by the link of unlawfulness and must be expelled from the body of evidence.

It is worth separating the plane of obtainment from the plane of processing. Even if the initial capture of the data was lawful, later processing through an opaque model can introduce autonomous defects: reuse for a purpose other than the one authorised, lack of judicial control over the algorithm, or the impossibility of verifying the chain running from raw data to conclusion. Each of these flaws is a point of challenge that counsel must articulate separately, never accepting the evidence simply because it comes from a technically sophisticated system.

Presumption of innocence and the ban on automated decision-making

Article 24 of the Constitution enshrines the right to the presumption of innocence and to a trial with all guarantees, and from both flows a requirement that the rise of predictive models makes urgent: no one may be convicted on the basis of an assessment that the court has neither performed nor can explain. The function of judging cannot be delegated. A system that assigns a risk score or flags a person as a likely perpetrator does not provide a proven fact but a statistical correlation; conflating the two empties judicial assessment of content and effectively transfers the decision to an algorithm that does not appear in court, gives no reasons and answers to no one.

The burden of proof lies with the prosecution, and that allocation does not shift because an unfavourable algorithmic prediction exists. The defence need not prove its client's innocence against the model's output; it is the prosecution that must establish, with valid and sufficient incriminating evidence, every element of the offence. When a predictive tool operates as an evidentiary shortcut, it subtly reverses that burden and creates a presumption of guilt that Article 24 forbids. Detecting and denouncing that reversal is a central task of the right to a defence.

Algorithmic bias adds a dimension of discrimination that connects with Article 14 of the Constitution. A model trained on biased historical data reproduces and amplifies those biases, so that the apparent objectivity of a figure conceals a structural prejudice. The defence must demand transparency over the training data, the variables used and the differential error rates across groups, because a prediction that systematically penalises a community is not neutral evidence but the mathematical formalisation of a discrimination that no court may take as the basis of its conviction.

Chain of custody, authenticity and integrity of digital data

The reliability of any predictive conclusion depends on the integrity of the data underpinning it, which is why the digital chain of custody is a first-order ground of challenge. Unlike traditional physical evidence, electronic data can be altered without leaving an obvious trace, so its evidentiary value requires proof that what was analysed is identical to what was originally seized. This is documented through cryptographic hash values, write-blocked forensic copies and a continuous record of who accessed the material, when and for what purpose. Any gap or silence in that traceability opens the door to arguing that the source of the prediction is neither authentic nor intact.

Authenticity takes on singular relevance with synthetic content. Images, audio or video generated or manipulated through artificial intelligence may be indistinguishable from the real thing to the naked eye, so no audiovisual material should enter the body of evidence without a technical analysis ruling out manipulation. The defence can and should require expert examination of metadata, internal file consistency and traces of artificial generation. It is also worth recalling that the criminal treatment of synthetic imagery has its own contours: Article 197.7 of the Criminal Code penalises the distribution of genuine intimate images obtained with consent and later disclosed without it, but its wording does not cover with equal clarity fully synthetic images in which no real capture of the victim exists, a gap that demands careful analysis of the proper legal classification.

With digital evidence, the guiding principle is that technological sophistication does not replace procedural rigour. A report that presents a result without explaining the full chain running from the seized medium to the final conclusion is, for defence purposes, an incomplete report. Counsel must reconstruct that chain link by link, identify where the documentation breaks and translate each gap into a reasonable doubt over the material's integrity, because without proven integrity there is no reliable evidence, however solvent the report's format may appear.

Right to a defence: contradictory expert evidence and effective challenge

The core safeguard against algorithmic evidence is the right to contest it on genuinely equal terms. It is of little use to say the defence may challenge a predictive model if in practice it has no access to its internal logic, its training data or its parameters. Opposing the black box is a legitimate legal argument: evidence that cannot be examined cannot be contested, and evidence that cannot be contested fails to meet the requirements of a trial with all guarantees under Article 24 of the Constitution. The defence must request the code, the technical documentation and the system's traceability, and formally record every refusal of access as a limit on adversarial scrutiny.

Contradictory expert evidence is the proper procedural tool to dismantle the appearance of infallibility. An independent expert can assess the methodology, set out false-positive rates, identify proxy variables that conceal discriminatory factors and show that the model's output depends on non-neutral design choices. It is essential to propose this expert evidence in due time and form, to frame the questions the opposing expert must answer at trial, and to ensure the technical debate takes place under conditions of immediacy and adversarial scrutiny before the court that must assess it, rather than resting on a one-sided report admitted without discussion.

Finally, the challenge must be exhaustive and tiered, abandoning no plane. One attacks first the lawfulness of how the data were obtained under Article 11.1 of the Organic Law of the Judiciary; then the integrity of the chain of custody; next the methodological reliability of the model and its biases; and, in every case, one invokes the presumption of innocence to recall that unresolved doubt favours the accused. This layered strategy prevents a single failed argument from dragging down the whole defence and keeps alive the requirement that it be a judge, and not an automated system, who decides on a person's guilt.