John William Waterhouse’s painting ‘Echo and Narcissus’ from 1903 has been adapted to our own time by an algorithm and a human being. What might happen if scientists could turn these clever programs into their research partners? | Image: Jonas Wyssen

The intelligent text program ChatGPT has been booming for the past few months, not least because it has been made available free of charge by the company that makes it, OpenAI. ChatGPT is also opening up completely new possibilities in the field of academia and has provided astonishingly good results: for those who are researching, writing or reviewing – and also for those who cheat. The scientific journal Nature has therefore been issuing rules for dealing with AI. One of them states that authors must be transparent in explaining the origins of any machine-generated statements. 

Mathias Grote is a historian of science at Bielefeld University. He sees this as a paradigm shift: “It seems as if we have reached one of those moments that signify a breakthrough, especially in terms of actually using AI”. He says that we experienced comparable moments when the Internet emerged, when Google’s search engine took off, and when the smartphone arrived. And it had been clear for some time that AI would one day arrive on the scene. Grote remembers the discussions that took place about AI in the early 2000s: “Back then, the debate was more about clarifying categories and about theoretical matters” – in other words, people were debating what might actually constitute artificial intelligence. Today, by contrast, it’s the technology itself that is at the centre of attention. Of course, economic and political motives are also playing an important role in the development of AI. Data corporations and major publishers, for example, are pursuing their monetary interests with the development of these text programs. 

Suddenly, everyone’s using it

The advent of ChatGPT-4 enables everyone to generate text – and it’s also being used to ‘write’ academic papers. Matthias Barton, a cardiologist at the University of Zurich, is appalled by these new tools: “Internet companies are making them available to students and researchers”. He is the co-editor of eLife, an open-access journal for biomedicine and the life sciences. “It’s not entirely clear who is behind these programs”, he says. He is also perturbed that they are being advertised by means of highly questionable statements, such as a claim that AI can enable a computer to think like a human without any human intervention. He actually believes ‘artificial intelligence’ is erroneous as a concept. It’s “fraudulent labelling”, he says, insisting that AI doesn’t really have any ‘intelligence’ at all. In perfectly banal terms, it involves nothing other than algorithms, especially when it’s called on to write a publication. It involves a process of depersonalisation, says Barton, and it’s relegating independent thinking to the back seat. “As a result, students and young scientists who are still at the beginning of their career are being enticed into using AI to ‘write’ scientific papers or parts of papers, but while giving their own name as the author”. 

New approaches are needed if we’re going to deal with this new form of cheating, as it’s already happening at university level. Lecturers will have to teach students the ethos of self-control, says Caspar Hirschi, a historian of science from St. Gallen. In order to justify actually checking a student paper, there has to be a strong suspicion of wrongdoing – such as major shifts in the language skills demonstrated within a paper. The sheer number of texts that would have to be checked makes it well-nigh impossible to subject students’ work to greater scrutiny. “Mutual trust is important”, says Hirschi, “but we also have to require students to sign a form confirming that their work is indeed their own, like the journal Nature does”. 

That is probably not enough to prevent cheating in journals. This is why Matthias Barton is in favour of introducing specific guidelines. “The Swiss Academies of Arts and Sciences have to become active, especially in medicine. And they have to be quick about it”. He would like all authors to confirm in writing that they haven’t used any AI in their work. Hirschi thinks this would be viable, though he doesn’t see it as a gamechanger. “The modern scientific system is based on trusting others to work legitimately”, he says, adding that experimental research has always functioned in this way. “Science has never been able to apply its control mechanisms across the board. If it had tried, the system would have collapsed”. 

Ban nothing, but exercise some control

Researchers have actually been familiar with similar problems for a long time now, because of papers being produced by ghostwriters. And they’ve long been fighting plagiarism too. Grote says that today’s AI programs nevertheless make it especially important that we should demand original thought. A digital watermark could be part of the solution, he says. This would entail affixing a technical ‘label’ to texts that would enable others to trace the sources used. But if we are going to learn to recognise forgeries created with the support of AI, it will also be necessary to practise the scientific use of AI itself. That in turn means it will have to become part of a student’s education. Grote believes that trying to ban it would be counterproductive. 

Using AI in scholarly publications also raises questions beyond merely declaring it. Matthias Barton believes that writing an abstract, for example, is actually part of the process of learning how to write scholarly texts – but it’s where the intelligent AI programs are already being used on a frequent basis. This learning experience is lost when people use AI instead. In principle, he says, publishing a paper is making a small contribution to the current state of knowledge and helps to advance scholarship overall. “But the universities are exerting a lot of pressure on people to publish, and this is extremely convenient for the publishers”. As a result, even doctoral students and postdocs can get tempted to work with AI programs instead of getting down to the task of reading the work of other scholars and thereby learning the art of scholarly writing. 

“Regrettably, this often happens in the field of medicine”, says Barton. “I know of cases where machine translations by DeepL have been copied one-to-one into manuscripts, though the authors naturally didn’t say so”. Here too, he says, there is a lack of the kind of regulations we need – the kind of regulations recently issued by the German Ethics Council and by various British universities. “They are necessary because human beings naturally like what’s convenient and will use whatever is available to them”. 

According to Hirschi, “You can get AI to write a text that’s not so bad at all”. AI can gather together the standard arguments, rather like Wikipedia. But such a text only represents one aspect of the current state of knowledge. It doesn’t always have references, and is devoid of any originality. What’s more, it isn’t able to conduct research or establish connections. Determining new methodological procedures is something that has to be carried out by researchers themselves. It can’t be delegated to AI. “This makes the gap even bigger between new research and simply writing a summary”.

No originality

Mathias Grote has a similar opinion of the current capabilities of AI. You can use it to create workaday texts swiftly and effectively and to carry out searches. “As long as the program can rely on facts that are already well-known and available, most of what it will do is correct – but not everything”. This is why such texts have to be edited further, reworked and corrected. At best, they can serve as the basis of something. “I can hardly imagine that AI alone will ever be able to make genuine scientific discoveries”, he says. He believes instead that AI can be used as a meaningful tool to retrieve standard information quickly and to perform routine tasks and calculations – just as we used to employ manuals, tables and calculators to that end. 

Despite all the criticism directed at the use of AI programs, Matthias Barton also sees opportunities. “If the algorithm is programmed well, then you can delegate searches to AI that can save you time and effort”, he explains. But it is absolutely necessary to check the results AI delivers, otherwise the danger of it outweighs the potential benefits. In medicine, there are limited advantages to using AI when collecting and processing large amounts of data. But if the use of AI is explicitly mentioned in a research proposal just because it’s currently in vogue, then there’s something wrong with the proposal itself. “It would be better to pause for a moment and consider whether we can really speak of progress here”. 

An aid to peer reviewing

AI is also already being used by academic journals in the peer review process. In his capacity as the co-editor of eLife, Matthias Barton does not find AI suitable for peer reviewing since it possesses no expert knowledge, nor can it evaluate the actual relevance of the literature cited in scholarly papers. eLife only uses proven experts for its peer reviews. Nevertheless, AI programs are already in widespread use as an aid in the peer review process. Many publishers use them to suggest potential reviewers to journal editors. “Many editors are happy if they can find any peer reviewers at all”, says Barton. 

But AI is also used in other aspects of peer review. Because even when a journal has found a potential reviewer, they are usually already overworked and have to divide their time between their regular work and their reviewing tasks. AI can help alleviate this stress by making suggestions based on the reviewer’s previous work, comments and interactions. Or at least, this is what the providers of these AI programs are promising. 

It doesn’t surprise Barton that companies like Elsevier or online publishers such as MDPI are relying on AI for peer reviews. “Many companies have long outsourced the publishing process to countries like China, India and Pakistan”, he says. Using extremely cheap labour can save publishers huge amounts of money – but at the same time, it means they are reducing their degree of responsibility for the content of the journals they are publishing. The more open the system, the more susceptible it is to things going wrong. By its very nature, every algorithm has its own limits and inaccuracies, and these then feed into the scholarly process.