Academics address risks of AI misuse

More than 100 researchers gathered in Beijing on Sunday to discuss how artificial intelligence tools can be used to assist, rather than distort, the research and publishing process.

The second Yuyuantan Conference on Research Integrity also released the Guideline on the Boundaries of AI-Generated Content Usage in Academic Publishing 3.0.

During the literature review stage, the guideline states researchers may use AIGC tools to collect references related to key terms or topics, categorize them and summarize conclusions to provide background material. But because AI-generated references may be fabricated or outdated, it warns that researchers must verify all citations manually and make humanled decisions. It also says that when using functions such as "deep thinking", researchers must ensure the credibility of source databases and cross-check any recommended references or conclusions to prevent AI hallucinations.

The guideline includes detailed recommendations across other stages of research, manuscript submission, peer review and post-publication, touching on data collection, statistical analysis and figure generation.

At the event hosted by the Institute of Scientific and Technical Information of China, Tan Tieniu, an academician from the Chinese Academy of Sciences, emphasized the importance of clarifying researchers' primary responsibility as AI users and standardizing disclosure of AI use in academic papers.

Tan, who also serves as secretary of the Communist Party of China Nanjing University Committee, called for expanded AI training within the research community to improve effective and responsible use.

"We should promote global collaborative governance, reach international consensus on AI usage norms and governance technologies, and advance Chinese standards," Tan said.

Elsevier, a leading international academic paper publisher, recently released its Research of the Future report, which found that Chinese researchers are the most active users of AI in research worldwide, with 69 percent reporting adoption — higher than the global average of 58 percent. About 66 percent of Chinese respondents said they believe AI will improve research quality.

"China is always ahead when it comes to technology and innovation," said Gemma Hersh, senior vice-president and sales and commercial director for primary research at Elsevier.

"China has always published a lot of research. Now what has happened over the last five to 10 years is that the quality of research has increased significantly, which is a huge achievement," she said, adding that AI adoption may have helped drive that shift.

The report, based on a survey of more than 3,200 researchers in 113 countries, also highlights uncertainties accompanying rapid technological change. Only 22 percent of global respondents consider AI trustworthy, while 45 percent said they lack sufficient AI skills.

Hersh said improving transparency and traceability is a shared need across the research community. She noted that Elsevier is providing stronger technical support through tools such as Scopus AI, ScienceDirect AI and the soon-to-launch Leap Space.

China's work to promote compliant and responsible AI use is setting an international example, she added.

"The release of the guideline clarifies the boundaries for academic publishing, research norms and disciplinary evaluation, identifies responsible entities, and helps researchers use AIGC as a scientific tool and assistant, providing clear guidance," said Geng Peng, an associate professor from the School of Communication at Tianjin Foreign Studies University.