The way our science is evaluated: Now is the time for change

By Avijit Banik

Like it or not, we all accept and at times succumb to the mandate of “publish or perish” in the current system of scientific evaluation. The number of publications, citation indices, complex log of impact factors, and corresponding and first authorships play a significant role in the matrix of hiring, promotion and success of a researcher. In today’s fast paced world, emphasis seems to be placed on the quantity rather than the quality of research. The continued pressure to publish research in peer-reviewed and recognized journals is leading to research malpractice and data fabrication, which have been reported many times in recent past.

But have we ever wondered what is the other side of the story? When there are so many journals and publishing houses contending and competing to be the reasonably best in the field, how do they remain unbiased and ethical during the peer-review process? Is there a robust system in place to check for potential gender bias, data fraud etc. in their review process? Do all the reviewers agree to the decision made by the editor towards the acceptance/rejection of research articles for publication? Is the “Impact Factor” meant to evaluate the progress of researchers or journal bodies?

Christophe Bernard, the Editor-in-chief of eNeuro, an open access journal of the Society for Neuroscience, strongly believes that his journal’s peer-review system has addressed most of the above stated concerns. Christophe is the current director of research at INSERM U1106 in the Institute of Systems Neuroscience in France where he researches brain dynamics in epilepsy. He was recently invited as a seminar speaker to the Department of Pharmacology at Emory University School of Medicine and led a workshop on “Everything you want to know about publishing papers”. The post-docs and graduate students from Pharmacology and Neurology had an opportunity for an hour long discussion on issues related to science publications such as peer review system, biasing in publishing, significance of raw data, citation, impact factors, and so on.

Christophe believes the current publishing system needs to change, especially how we evaluate science today, otherwise a system crash is inevitable in the near future. And we, the younger generation have to take a lead on this. Unless we regularly question the current practice, we will be at the mercy of this evaluation process formulated several decades back. He strongly believes that citation, h-index and impact factor matrices are not an ideal medium to evaluate the scientific quality of a researcher aspiring for transition or promotion in his/her academic career.

These publication parameters can be easily manipulated either by the researchers themselves or by the journal editors. To understand the system, one needs to understand how impact factors (IF) are calculated which is IF = Total no. of Citations/Total no. of Publications in the last two years. By controlling the number of publications against time, journals can pump up their IF for certain years. In most of the cases, the published articles come into the public domain in the web version much earlier than the printed version and at times this gap between the web version and printed version is more than a year.  Several renowned journals take advantage of the early citations from online articles while the date from the printed version is factored into the IF calculation for last two years. This gives an uneven advantage to the number of citations for articles which still have not made their way into the printed copies. A group of well networked researchers can also accelerate their citation indices by mutually agreeing to cite each other’s publications in their abstracts published in conference proceedings.

Christophe has taken some serious measures to counter this skewness in the system. He introduced a “double blinded peer-review” system for eNeuro in which authors for a submitted article are masked from the reviewers during review process. This largely eliminates gender bias and favoritism from the review process and the articles are more likely to be measured on their scientific content without weighing gender, reputation, institution, or country of origin of the authors. eNeuro also publishes reviewer’s comments associated with the accepted articles to counter the disagreement between the reviewers in the process of accepting an article for publication. Christophe emphasizes publication of raw data and technical details of statistical analysis as supporting information to increase the chance of reproducibility in future. In one of his recent editorials in eNeuro, he mentioned, “my personal interpretation is that the double-blind review process minimizes gender bias during the evaluation of the paper. It avoids adding an additional layer of bias to the dissemination of research findings. Consequently, most, if not all, journals should adopt it.” Before he signed off from our discussion, he was happy and proud to mention that the Nature publishing group recently started following their double blind review process to reduce bias in the review system.

Picture1

Left) Dr. Avijit Banik and Dr. Christophe Bernard. Right) Dr. Bernard speaking to students.

Reference:

Christophe Bernard. Editorial: Gender Bias in Publishing: Double-Blind Reviewing as a Solution? eNeuro 26 June 2018, 5 (3) ENEURO.0225-18.2018; http://www.eneuro.org/content/5/3/ENEURO.0225-18.2018

Avijit Banik (LinkedIn profile) is a Post-doctoral Fellow in the Department of Pharmacology at Emory University.

Advertisements

About EmoryPDA

We are the Postdoctoral Association at Emory University. We are dedicated to improving the postdoctoral experience for ALL fellows at Emory University.
This entry was posted in Blog. Bookmark the permalink.