If only an algorithm could predict which publications out of many would one day be important. | Photo: sear-greyson-K-ZsC7YdJ6Y-unsplash

The flood of articles in specialist journals is a strain on the scientific ecosystem in which researchers assess the work of their colleagues. This is also why citation-based metrics are repeatedly used, despite widespread criticism that they “provide only imperfect, inconsistent and easily manipulated measures of research quality”, in the words of James Weis and Joseph Jacobson. These two researchers from the Massachusetts Institute of Technology have a solution: artificial intelligence. It can calculate how big an impact a scientific publication will have in a specific field in a few years’ time.

Using data from biotechnological journals from 1980 to 2019, they have been able to determine correctly, in 19 out of 20 cases, whether or not a publication would land in the top five percent. They used 29 indicators that primarily describe the number of citations and the networks of the authors.

It didn’t take long for researchers to criticise them on Twitter: “When a measure becomes a target, it ceases to be a good measure” is a law from economics that is quoted by Anders Sandberg of the University of Oxford. Daniel Koch of King’s College, London, wrote: “Unfortunately, once again ‘impactful’ is defined mostly by citation based metrics, so what’s ‘optimised’ is scientific self-reference”.

Andreas Bender, the director of the Innovation Campus in Berlin, said: “It will only serve to perpetuate existing academic biases into the future” – such as biases regarding gender and background, as Julia Gala de Pablo of the University of Tokyo has remarked: “I would never want my clearly Hispanic-sounding female name analysed with a software like this for funding allocation!”