On the grapevine
Non-reproducible because of missing data
Often not even the editor of a specialist journal can get access to the raw data for articles submitted.
Early this year, Tsuyoshi Miyakawa, the editor-in-chief of Molecular Brain, published an analysis of his decisions about the manuscripts he had been sent. He has examined 180 of these since early 2017. In 41 cases, he asked the authors to make their raw data available. To his surprise, this resulted in 21 manuscripts being withdrawn. According to Miyakawa, this means “asking for their raw data drove away more than half of the manuscripts”. Of the 20 that remained, he rejected 19 because their raw data was inadequate. “This means that 97 percent of 41 manuscripts were unable to provide proper raw data to support their results when requested by an editor. This suggests that such raw data – in at least some of these cases – had never existed at all”.
Miyakawa now believes that absent raw data, or invented, phoney data, may be the reason why many scientific studies cannot be reproduced, alongside further inappropriate practices such as postulating hypotheses after the results have become known (HARKing), adapting statistical analysis parameters after the fact (p-hacking), and publishing positive results selectively (publication bias). He is now insistent: “Given the fact that every scientific study ought to be based on raw data, and that having enough data storage space is no longer an issue, journals should in principle insist that authors should place their raw data on public access in a database or in a journal, once they’ve published their article”. This would increase the reproducibility of their results and strengthen public trust in science.