This article was originally published on RealClearScience.
John Ioannidis is (in)famous in the scientific community. Using straightforward logic and statistics, he convincingly demonstrated that most published research articles are wrong. This is not because scientists are liars and crooks, but because studies often do not have large enough sample sizes or are testing unlikely hypotheses. Ioannidis’s revelation sent a shock wave through the biomedical community. Partially in response to his findings, biomedical scientists began to embrace reforms in scientific publishing, such as using more open access journals and publishing replications and negative data.
Now, in a new paper in PLoS Medicine, Dr. Ioannidis proposes additional reforms. Some of the more interesting ones include:
Registration of studies. Clinical trials already do this. (See ClinicalTrials.gov.) This would allow researchers to monitor ongoing studies. Others have proposed that all registered studies should be accepted for publication upon their completion, regardless of the outcome of the experiment. This would eliminate “publication bias,” the phenomenon in which only “sexy” results are published and negative (or uninteresting) results are ignored.
Adoption of better statistical methods.It is not a big secret that biologists are bad at math. (Well, except for John Ioannidis.) Papers with a lot of mathematical equations are avoided by biologists. The heavy mathematical lifting required in some biomedical fields, such as epidemiology and genomics, is outsourced to biostatisticans. Dr. Ioannidis suggests more stringent thresholds for statistical significance. That is certainly necessary, but there also should be a requirement for all biomedical PhD students to take courses in biostatistics.
Improvements in peer review. Though Dr. Ioannidis does not offer specific details, one group is strongly advocating post-publication peer review. F1000Research publishes papers along with their expert reviews, which are not allowed to be anonymous.
Consideration of stakeholder interests and modification of incentives. Members of the scientific community place different values on research. For example, professors want research that is publishable, while industry wants research that is profitable. Dr. Ioannidis identifies four interests that need to be considered: publishability, fundability, translatability, and profitability. Furthermore, he proposes a radical change in incentives, such as eliminatingacademic ranks (e.g., tenure).
Scientific publications are on the cusp of a dramatic shift. Thanks to the efforts of Dr. Ioannidis and others like him, the monolithic biomedical establishment is beginning to embrace change. If only the rest of academia was so reflective and self-critical.
Source: Ioannidis JPA (2014). “How to Make More Published Research True.” PLoS Med11(10): e1001747. doi:10.1371/journal.pmed.1001747