In 2017, a public health study found a curious association between stroke and diet soda consumption—a result that had significant limitations and required further research. But this study nonetheless became a sensation in the mainstream media, where it was widely mischaracterized as proof that diet soda caused strokes, spreading false alarm.
Today, AI tools are starting to make the same kind of mistake.
For Duke Kunshan University alumnus Nicholas Peoples, a resident physician in emergency medicine at Massachusetts General Hospital, mistakes like these could have serious consequences for patient care, especially as AI becomes an increasingly common source of medical information.
“AI is unusually prone to providing wrong answers that sound right and are delivered with enormous confidence,” Peoples said.
In a commentary published August 24 in npj Digital Medicine, Peoples and three co-authors propose two ways to reduce such errors. The first is a plain-language abstract for human readers, called “FACTS,” to make the nuances of a study’s findings clearer to non-expert audiences, such as journalists. The second involves putting hidden instructions for AI into published research, which they call a “SIMcard,” to help steer AI tools away from providing an incorrect interpretation of the results. In both cases, their goal is to help AI systems and human readers understand not only what scientific findings show, but also what conclusions the evidence cannot support.
The team includes Bing Luo, assistant professor of data and computational science at DKU; Lijing Yan, professor of global health at DKU; and Ken Milne, an emergency medicine physician affiliated with the University of Oxford and Western University in Canada. Peoples conceived the two proposed frameworks and led the writing of the paper.
Giving AI a clearer guide to scientific findings
The idea for SIMcard grew out of Peoples’ interest in “prompt injection,” a technique that involves putting hidden instructions in documents to influence an AI system’s responses. Rather than manipulating AI to produce a misleading result, Peoples wondered whether researchers could provide structured information to help it interpret scientific findings more accurately.
SIMcard stands for “Structured Interpretive Metadata.” It would provide AI systems with standardized, machine-readable information about a study, including its design, main findings, limitations and conclusions that should not be drawn from the evidence.
The information would be embedded in a study’s metadata, which currently serves largely to help computers identify and retrieve publications. Under the proposal, researchers would prepare the interpretive information and journal reviewers would evaluate it alongside the scientific manuscript.
Luo offered a simple example: An observational study might find an association between diet soda consumption and stroke. An AI system could incorrectly summarize the finding as “diet soda causes stroke,” overlooking other possible explanations and the distinction between association and causation.
A SIMcard could instead provide a structured record of the study’s findings and limitations, helping AI systems distinguish what the evidence supports from what it does not.
The goal is not to make AI infallible.
“It should be understood as a guardrail, not a guarantee,” Luo said.
The second framework, FACTS, or “Framing Appropriate Claims & Their Scope,” is designed for human readers. Presented in plain language, a FACTS label would explain a study’s main claim, whether its findings establish causation or association, the context in which the research was conducted, key limitations and interpretations to avoid.
Both frameworks remain proposals rather than tested solutions. The authors say further research is needed to determine whether they improve interpretation across different studies, AI models and prompts. Widespread adoption would also require coordination among researchers, journals, indexing services and AI developers.
A longstanding problem, amplified by AI
The authors’ concern builds on a problem that predates generative AI: scientific findings can become distorted as they are cited and repeated.
In a 2023 article published in The BMJ, Peoples, Yan and Duke University professor Truls Østbye discussed how inaccurate citations can spread misleading claims through scientific literature.
One example involved a 1980 letter published in the New England Journal of Medicine. The letter reported that addiction was uncommon among hospitalized patients who received opioid medications for short periods of time. Over the following decades, however, it was repeatedly cited as evidence that addiction was rare with long-term opioid use, a conclusion the original letter did not establish.
A 2017 analysis identified 608 citations of the letter and found that many failed to acknowledge the hospital setting in which the patients received treatment. The researchers suggested that the misleading citations may have contributed to the North American opioid crisis by downplaying concerns about addiction risk.
The example illustrates how an initially narrow finding can become a widely accepted claim through repeated misinterpretation. AI could accelerate that process as more people rely on automated summaries rather than reading the original research.
Implications for global health
The paper also points to a broader global health concern: AI-mediated interpretation could deepen existing inequalities in access to scientific knowledge.
Biomedical research is disproportionately conducted in high-income countries and published primarily in English, while people around the world increasingly rely on AI systems to access and summarize that research.
When findings are interpreted for audiences in different languages and health systems, missing context or inaccurate generalizations could make the evidence appear more widely applicable than it is. A study conducted in one population, for example, may not support the same conclusions for patients elsewhere.
“AI has the potential to make medical research more accessible to people around the world. But access alone is not enough. We also need to ensure that scientific findings are interpreted accurately and in the right context, particularly when research conducted in one population is applied to another,” Yan said.
An interdisciplinary collaboration rooted in DKU
For Peoples, the project also reflects the interdisciplinary education and relationships he developed at DKU.
He joined DKU’s Global Learning Semester in spring 2015 and later earned a Master of Science in Global Health in 2018. During his time there, he worked with Yan on a project involving digital health interventions to improve hypertension care in rural Nepal.
Peoples credits DKU with strengthening his commitment to rigorous scientific thinking, an approach he considers essential in medicine and research.
“I got that foundation at DKU,” he said.
He has maintained research collaborations with DKU colleagues over the years. In the latest project, Luo’s expertise in AI systems and structured data complemented Peoples’ clinical perspective, while Yan and Milne contributed critical review and revisions to the manuscript.
“DKU’s interdisciplinary environment made this collaboration possible,” Luo said.
