Illustration: Johanna Schaible

The case of Amanda Knox made the headlines the world over. She was an exchange student from the USA who in 2009 was sentenced to 26 years in prison in Italy for the brutal murder of her British flatmate. Above all, it was the results of forensic analysis that had incriminated her. Traces of DNA from both Knox and the victim were found on the presumed murder weapon, a kitchen knife. But two years later, an appeal court annulled the verdict after judging that the DNA results were in fact unreliable. After being convicted a second time, Knox was finally acquitted in 2015.

“What lawyers know about forensics typically comes from TV series and films”. Joëlle Vuille

Joëlle Vuille is a legal expert and a criminologist. She and her colleagues can name a whole list of mistakes made by the forensic scientists in the Knox case. The investigators hadn’t always changed their protective gloves while gathering their clues; in one analysis, the amount of DNA used was too small for the apparatus in question; and the scientists involved even drew questionable conclusions when interpreting the samples that contained the DNA of several people.

There has still not been any comprehensive study of the number of cases in which people have been wrongly convicted on the strength of evidence from scientific experts. But this number is presumably much lower in Europe than in the USA, not least because of the differences in their judicial systems. “In Anglo-American countries, the experts often work on behalf of the prosecutors, and so sometimes feel compelled – whether consciously or not – to reach incriminatory results”, says Vuille. In continental Europe, however, the experts usually work on behalf of a neutral magistrate, and thus have no obligations to either side.

Fatal communication problems

Nevertheless, as Vuille explains, even more cases have come to light in Europe in recent years where forensic reports have led to a miscarriage of justice. As part of a study at the University of Neuchâtel, she is collecting just such cases and is looking for what causes them. “One problem is that several scientific methods are not reliable – as has been proven in recent times”. One classic example, according to Vuille, is the analysis of bite wounds. Attributing a specific set of teeth to a bite wound in the skin of the victim is something that depends solely on the experience of the expert in question. There are essentially no scientific studies about it, and no methods have been properly validated. Even the analysis of shoe prints, tyre tracks and comparisons of hair and fibres have come under criticism in the past years. And as the case of Amanda Knox has shown, even a seemingly reliable method such as DNA analysis can rapidly reach its limits.

a + b + c = jail
The nurse Lucia de Berk was present at nine of 26 serious accidents involving patients that occurred in three Dutch hospitals between 1997 and 2001. Not surprising then, that the authorities suspected foul play. At de Berk’s trial, the specialist in criminal psychology and former statistician Hank Eiffers was called to testify. He estimated the chance of her presence at those nine incidents being a coincidence at one in 342 million. So convincing was his statistical evidence that, on 24 March 2003, the court in The Hague sentenced de Berk to life in prison.

Mathematicians later questioned this expert testimony. In fact, Eiffers was wildly off the mark. Not only was the probability of the coincidence only one in 26, but he should, above all, never have been allowed to present to the court evidence uncorroborated by the probabilities of alternative scenarios. This well-known issue is known as the ‘prosecutor’s fallacy’, according to which a lottery winner is unlikely to have cheated, despite the one in 130 million chance of winning…

In 2008 experts petitioned for the case to be reopened. After many years in prison, de Berk was returned to the dock, and on 14 April 2010 she was acquitted. In the original case, a poorly presented and incorrect calculation had blinkered a court and blindsided a nurse whose only crime had been showing up to work. dsa

Vuille believes that by far the biggest problem, however, lies in failures in communication between the courts and the scientific experts. “On the one hand we have the lawyers who have no clue about science, and on the other hand we have the scientists who have never learnt to express themselves in a way that lawyers will understand”. Above all, she finds fault with the fact that lawyers don’t have to attend any courses in the natural sciences during their legal training. “What lawyers know about forensics typically comes from TV series and films”. And they tend to depict science as all-knowing and infallible.

“As a matter of principle, we don’t use any specialist terms in our reports”.Eva Scheurer

For Eva Scheurer, the head of the Institute of Forensic Medicine of Basel-Stadt, communicating with lawyers is an important part of her work. She and her colleagues write some 14,000 reports and expert opinions every year in the fields of forensic medicine, forensic genetics, forensic chemistry and toxicology, usually on behalf of the public prosecutor’s office and the police. Scheurer sees her role as very much that of a ‘translator’ between scientists and lawyers. “As a matter of principle, we don’t use any specialist terms in our reports. Instead, we describe the circumstances in everyday language”.

No percentages in reports

If necessary, she also provides basic knowledge. For example, providing an introduction to metabolic processes can help a judge to a better understanding of the results of a toxicological analysis. In expert reports for criminal proceedings, she also avoids expressing her results in terms of percentages: “If I say that there is a 45 percent or 55 percent certainty that a specific object is the weapon used to commit the crime, it actually says nothing at all. The court’s questions can’t be answered like that”. If one of several possible objects could be the weapon, then she describes in words which of them are more or less plausible than the others. Because experts are somewhat seldom invited to take the stand in Swiss courts, she usually does not get any feedback about whether the court has understood and interpreted her findings correctly. This is why she is planning workshops in future, where lawyers will read anonymised reports and give the forensic doctors direct feedback.

Tracking down miscarriages of justice
The findings of Innocence Project, a non-profit-organisation in the USA, demonstrate that forensic science can help to reveal miscarriages of justice. Since it was founded 25 years ago, 350 wrongly convicted people have been let out of prison, twenty of whom have been saved from execution. In every case, their innocence was proven by the forensic analysis of DNA traces that were often decades old.

By contrast, the European Network of Forensic Science Institutes (ENFSI) argues against the consistent use of numbers and probabilities. “In our opinion, this could prevent misunderstandings between the courts and the scientists”, explains Tacha Hicks Champod, a forensic scientist at the School of Criminal Justice of the University of Lausanne. According to the ENFSI Guideline for Evaluative Reporting in Forensic Science, published in 2015, scientific experts should always formulate their findings as a likelihood ratio in the light of at least two possible scenarios. For example, it is “in the order of 2,000 times more probable given the proposition that ‘Mr Suspect’s left shoe is the source of the mark’, than given the proposition that ‘An unknown person’s left shoe is the source of the mark’”. On the other hand, statements should be avoided such as: “The shoe print is consistent with the shoe of the suspect”. But such a procedure becomes difficult in areas of forensic science that are founded on empirical values and not on standardised methods. “In many classical disciplines we have the knowledge, but we lack structured data”, admits Hicks Champod. “That’s why it’s important to carry out more research in precisely these areas”. At the School of Criminal Justice, for example, there is a project that aims to develop a solid statistical method for comparing signatures.

“There’s always a certain wiggle room when interpreting information”. Eva Scheurer

Regardless of how different their approaches might be, everyone is in agreement that transparency is the top priority for a forensic report. This includes a logical structure, a precise description of the procedure, naming possible sources of error, and being open about the limits of science. “You can never exclude something 100 percent”, says Eva Scheurer, “because there’s always a certain wiggle room when interpreting information”. That’s why she’s also comfortable if a judge asks for a second opinion that might interpret her findings differently. “It’s not about my being right, it’s about finding the truth”. It’s then up to the judge to decide which report is deemed more important for reaching a verdict. In our legal system, after all, guilt and innocence are ultimately decided by the courts, not by scientists.

Yvonne Vahlensieck is a freelance science journalist who lives near Basel.

CC BY-NC-ND