Artificial-Intelligence (AI) chatbot called ChatGPT has written convincing fake research-paper abstracts that scientists were unable to spot, a new research has revealed.
A research team led by Catherine Gao at Northwestern University in Chicago used ChatGPT to generate artificial research-paper abstracts to test whether scientists can spot them.
According to a report in the prestigious journal Nature, the researchers asked the chatbot to write 50 medical-research abstracts based on a selection published in JAMA, The New England Journal of Medicine, The BMJ, The Lancet and Nature Medicine.
They then compared these with the original abstracts by running them through a plagiarism detector and an AI-output detector, and they asked a group of medical researchers to spot the fabricated abstracts.
The ChatGPT-generated abstracts sailed through the plagiarism checker: the median originality score was 100 per cent, which indicates that no plagiarism was detected.
The AI-output detector spotted 66 per cent the generated abstracts. But the human reviewers didn’t do much better – they correctly identified only 68 per cent of the generated abstracts and 86 per cent of the genuine abstracts.
They incorrectly identified 32 per cent of the generated abstracts as being real and 14 per cent of the genuine abstracts as being generated, according to the Nature article.
“I am very worried,” said Sandra Wachter from University of Oxford who was not involved in the research.
“If we’re now in a situation where the experts are not able to determine what’s true or not, we lose the middleman that we desperately need to guide us through complicated topics,” she was quoted as saying.
Microsoft-owned software company OpenAI released the tool for public use in November and it is free to use.
“Since its release, researchers have been grappling with the ethical issues surrounding its use, because much of its output can be difficult to distinguish from human-written text,” said the report.
Comment here