Science Thursday [Warning: Requires you to put on your statistician hat] – February 27, 2014

A

Nature

article entitled, “Scientific Method: Statistical Errors,” discusses how despite there being many potential methods for analyzing scientific data, not every method can be applied evenly to every circumstance, and that interpretation of data and the reproducibility of experiments

can create numerous problems in research

. The issues raised in the article are applicable to federal efforts to validate and develop standards for forensic disciplines, highlighting the need for statisticians in this research.

 

In any experiment, scientists test if a variable, say, a specific medicine, can alter the outcome of the study group — the group that is under the influence of the variable — in comparison to the control group — the group not experiencing the variable. The “P value,” long considered the “golden standard” of statistical methods, is often used to determine whether the difference caused by the variable was significant or due to chance. According to

Nature

, statisticians warn that P values are not reliable and were not designed to be definitive. The P value calculation is “

based on the assumption that there is no real effect

.” At best, the P value can “summarize the data assuming a specific null hypothesis. It cannot work backwards and make statements about the underlying reality. That requires another piece of information: the odds that a real effect was there in the first place.”

 

Fortunately, other statistical methods are available and are currently being considered within forensic science.

 

An article by


Science News


, suggests that scientists should rely on statisticians to interpret experimental data in order to present cogent, defendable conclusions because as a recent

PLOS Medicine

article suggests, most research findings could be false due to bias, issues with experimental designs, and

poorly drawn statistical inferences

.

 

Current efforts to improve the scientific and statistical foundations of forensic science recognize this need.

The National Commission of Forensic Science

, tasked with developing policy recommendation for the U.S. attorney general, has a statistician as a commission member. The Organization of Scientific Area Committees (OSAC), administered by the

National Institute of Standards and Technology

(NIST), will include statisticians on various standard developing committees.

 

Both the National Commission of Forensic Science and NIST should be commended for including statisticians in this important work within forensic science.

 

Luckily, the American Statistical Association has become involved in

forensic science advocacy

and is making efforts to engage more and more statisticians in forensic science. Statisticians can help forensic scientists determine which method is most applicable to the forensic science discipline at hand based on the underlying assumptions made by both the discipline and the statistical analysis.