Probative Value of Forensic Science Conclusions Should be Based on Empirical Data, Not Subjective Impressions

The top statistical society in the United States issued guidelines for the statistically sound expression of the probative value of forensic evidence. The 2009 National Academy of Sciences report on Strengthening Forensic Science in the United States: The Path Forward and the 2016 President’s Council of Advisors on Science and Technology report on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods have encouraged the use of empirical data and quantitative analyses as the basis of probabilistic statements that link a piece of evidence from a crime scene to a potential source.

The misapplication of forensic science is a contributing factor in 45% of the 362 wrongful convictions overturned by DNA evidence. Among the problems that account for the misapplication of forensic evidence is misleading forensic testimony that overstates or exaggerates the significance of similarities between evidence from a crime scene and evidence from an individual, or oversimplifies the data. In 2015, the FBI and U.S. Department of Justice formally acknowledged that FBI examiners gave flawed testimony regarding the probative value of microscopic hair comparison evidence in over 95% of the first 268 trials in which the evidence was used to ascertain the guilt of a defendant.

“Past errors provide us with opportunities to work with the forensic science community to improve the process and strengthen the use of forensic evidence,” said ASA president Karen Kafadar. “This statement is just one step in that direction, but we hope it will be an important step.”

The American Statistical Association (ASA) guidelines build upon a document guided by the late and esteemed statistician, Dr. Stephen Fienberg, at the National Commission on Forensic Science, but was not passed before the Department of Justice declined to renew its charter. In the absence of databases and empirical evidence, the interpretation of forensic evidence has been based on an examiner’s training and experience. The ASA statement explains why personal impressions are not sufficient for communicating scientific forensic conclusions to the investigators and fact finders who must make critical decisions based on that information.

“The ASA statement being adopted by the entire organization is extremely important because the ASA has an extraordinary reputation and a very large membership,” said Peter Neufeld, co-founder of the Innocence Project, the nonprofit legal organization that exonerates the wrongly convicted through DNA testing and reforms the criminal justice system to prevent future injustice.

“We have plenty of first-hand encounters where forensic examiners relied on their subjective impressions and personal experience instead of statistical data to explain the value of the evidence,” added Neufeld. “As a consequence, there were serious miscarriages of justice, where factually innocent people lost decades of their lives and some of them were sentenced to death. Hopefully policymakers, judges and people who really care about the strength of evidence used to make decisions about life and liberty will take notice and do everything they can to apply rigorous scientific and statistical principles in the future.”

Prominent members of the ASA also serve as lead investigators for the Center for Statistics and Applications in Forensic Evidence (CSAFE), a center of excellence sponsored by the National Institute of Standards and Technology. This research center “works to build a statistically sound and scientifically solid foundation for the analysis and interpretation of forensic evidence to grow competence in the forensic science and legal communities,” providing the foundations that would support valid and reliable interpretation of forensic evidence in the criminal justice system.

To read the ASA guidelines, click here.

To learn more about CSAFE, click here.