Forensic science isn't 'reliable' or 'unreliable' – it depends on the questions you're trying to answer

After recent criticism in the US and the UK, forensic science is now coming under attack in Australia. Several recent reports have detailed concerns that innocent people have been jailed because of flawed forensic techniques.

Among the various cases presented, it is surprising that the most prominent recent miscarriage of justice in Victoria did not rate a mention: the wrongful conviction of Farah Jama, who was found guilty of rape in 2008 before the verdict was overturned in 2009.

This omission is not entirely unexpected. The forensic evidence in the case against Jama was DNA. Despite this fact, the recent media comments have re-emphasised the view that DNA is the gold standard when it comes to forensic techniques. Justice Chris Maxwell, president of the Victorian Court of Appeal, said:

…with the exception of DNA, no other area of forensic science has been shown to be able reliably to connect a particular sample with a particular crime scene or perpetrator.

How can the same technique simultaneously be the forensic gold standard and contribute to such a dramatic miscarriage of justice? Is forensic science so unreliable that none of it should be admissible in our courts? Of course not, otherwise the criminal justice system would be left relying on much less reliable evidence, such as witness statements and confessions.

Read more: Get real, forensic scientists: the CSI effect is waning

Evidence in context

It makes no sense to assess the reliability of any forensic technique in the abstract. A forensic method is only “reliable” as far as it helps answer the particular questions asked in the context of a particular case. Asking the wrong questions will undoubtedly deliver the wrong answers, even if the best and most fully validated forensic method is applied.

Conversely, some forensic methods are perceived by some commentators to have less intrinsic value or even questionable reliability. But these methods might yield the answer to a crucially relevant question.

A typical example would be an incomplete shoe mark of poor quality left at a crime scene. It might not be possible to assign this mark to a specific shoe, but it might be enough to exclude a particular shoe or to identify the direction in which the perpetrator walked.

Forensic science is much more than merely applying methods or conducting tests – success also depends on the ability to identify and answer a relevant question.

A forensic science system is not like a clinical laboratory, processing samples and producing results for prescribed tests. Rather, good forensic science requires collaboration between investigators, scientists and other stakeholders. The focus should be resolving judicial questions using a scientific approach.

What matters most is the detection, recognition and understanding of the traces left by individuals during an alleged crime. This a much more complex issue than simply deciding whether or not a particular forensic method is deemed “reliable”.

Complex process

Forensic science is much less cut-and-dried than television dramas might suggest. When a DNA swab or a shoe mark lands on a forensic scientist’s lab bench, it has already gone through many steps, each with their own uncertainties.

These uncertainties are unavoidable, because forensic traces typically represent the aftermath of a chaotic event. The only option is to manage these uncertainties through a better understanding of how these traces are generated, persist, degrade, interact with each other, and how the information they hold can be interpreted.

The debate about the reliability of forensic science is not new. It illustrates a more fundamental issue: the lack of understanding of forensic science among the general public (who are potential jurors), and even among highly reputable law practitioners and non-forensic scientists.

Legacy of reform

The high-profile 2009 US National Academy of Sciences report and the 2016 Obama Administration report, both of which criticised some uses of forensic evidence, prompted an international reaction and several reviews of forensic practices.

They justified more empirical research to support some forensic conclusions. These improvements have been occurring in Australia for some years under the leadership of the National Institute of Forensic Science and through several academic research programs. And the recent UK House of Lords enquiry into the state of forensic science in England and Wales identified the Australian forensic science model as a leading example.

However, these reports excluded crime scene management from the scientific domain. They provided limited guidance about the challenging topic of interpretation of forensic evidence. This is disturbing because these are the two areas that require most attention if we are serious about improving forensic science outcomes.

Read more: 'This is going to affect how we determine time since death': how studying body donors in the bush is changing forensic science

As the recent media coverage has shown, evidence interpretation remains a sore point between the legal and scientific communities. Where is the boundary of the responsibility of science versus the law? The fact that the legal community poorly understands forensic evidence is undoubtedly a shared responsibility. Shifting the blame onto forensic science will only exacerbate the problem.

If we think this is all too hard with traditional physical evidence, how does the criminal justice system expect to cope with our rapidly evolving digital society? Digital evidence is typically harder to assess than physical evidence in terms of volume, variety, rapidity, and privacy issues.

Better education, research and collaboration will form a large part of the answer. They will induce a better understanding of forensic science and its fundamental principles, so it can serve justice with confidence.