AssociationAI / AI Literacy
AssociationAI team Published

Article

Beginner

An AI Summary of a Study Can Claim More Than the Study Found

AI summaries of studies were nearly five times more likely than expert write-ups to broaden the finding. What that means when staff draft a member research roundup, and why asking the tool to be accurate did not fix it.

AI Literacy Research Communications Member Trust

In a study published in Royal Society Open Science, researchers at Utrecht University and Western University compared AI summaries of 100 medical articles with summaries that experts wrote for NEJM Journal Watch, and found the AI summaries “nearly five times more likely to contain broad generalizations.” Across the whole study they checked 4900 AI-written summaries against the original papers.

That number lands on a familiar desk task. Communications staff often paste new abstracts into an approved chatbot and ask for short write-ups for a monthly research roundup. The drafts sound fluent, and the topic is usually right. What shifts is the strength of the claim: a result that “was associated with” an outcome in one sample becomes something that “improves” outcomes, past tense becomes present, and a sentence appears about what members “should” do.

Consider the monthly roundup at the Invented Ostrevale Society of Respiratory Therapists. This is a teaching example, not a case study. A coordinator asks a chatbot to turn six new papers into 60-word summaries. One paper reported that a training program “was associated with” fewer missed workdays in its cohort. The AI draft says the program “improves outcomes” and that “therapists should” add it to every visit.

What the researchers actually compared

The paper, Generalization bias in large language model summarization of scientific research, tested 10 models. The authors also collected “expert-written summaries from NEJM Journal Watch.” The comparison that produced “nearly five times” used summaries from GPT-4 Turbo (the API and the chat interface), ChatGPT-4o (the chat interface), and DeepSeek (the chat interface) of “100 full-length scientific (medical) articles that had corresponding human-authored summaries published in NEJM JW.”

The published abstract says: “In a direct comparison of LLM-generated and human-authored science summaries, LLM summaries were nearly five times more likely to contain broad generalizations (odds ratio = 4.85, 95% CI [3.06, 7.70], p < 0.001).” That figure is an odds ratio in the paper’s own words, not a share of the summaries.

The three ways a summary overshoots

The authors tracked three kinds of broadening.

Generic claims drop the limiter. They defined these as “present tense generalizations that do not have a quantifier (e.g. ‘many’) in the subject noun phrase.” A finding about “this cohort” becomes a finding about the program itself. They also found that “the most frequent transitions from a narrow generalization in the original text to a broader generalization in the LLM summary were transitions from quantified generalizations to generics.”

A tense change can do the same work. “When past tense result claims from an original text are turned into present tense in the summary, a broader generalization is conveyed than the author(s) of the original text may have intended.”

Advice is a further step. “When descriptive result claims are summarized such that action guiding recommendations are communicated, this involves a broader generalization than that found in the summarized text.” In a member newsletter, that is the “therapists should” line nobody asked the authors to supply.

Soft wording is not a reliable alarm. Broadened claims “often contained hedges and subtle scope extensions (e.g. ‘suggests’, ‘may’, ‘can lead to’)” and still went further than the original text. A pass that only checks whether the numbers add up can miss this; we covered that separate habit in rechecking every number in an AI summary.

Teaching sketch: a paper sentence about a finding in one cohort sits next to an AI summary that drops the limiter, switches to present tense, and adds advice.

Telling the model to be accurate did not help

The researchers tried the obvious extra instruction. When they told the models “do not introduce any inaccuracies when summarizing,” the summaries were “about twice as likely” to contain generalized conclusions as with a plain request (OR = 1.90). A prompt that sounds like a quality check can still leave you with a roundup that overstates the papers.

What a roundup editor can hold onto

Keep the paper’s limiter, tense, and “associated with” wording. Do not add a “members should” line unless the authors themselves recommend it. Ask the person who reviews the roundup to compare each sentence with the abstract’s conclusion, not only with names and figures.

Sponsored research updates that run in member channels deserve the same comparison. The society’s name sits on that channel, so a summary that overstates a study becomes the society’s claim even when someone else supplied the paper. That application is our reading; the study showed that summaries broadened claims beyond the original texts.

If the first worry is that the tool retrieved the wrong page, start with checking every claim against the retrieved passages. To score how far a correctly sourced summary has traveled past the paper, use the companion tutorial and check an AI research summary for claims stronger than the study. Our ten-step fact-check routine already includes putting the hedges back.

Limits worth saying out loud

“Not all generic, present tense, or action guiding generalizations” are a problem, the authors wrote, and they “did not assess whether the generalizations in human-authored texts were warranted.” The human side of the five-times comparison is also narrow: “NEJM JW authors are domain experts writing for an expert audience.” The study did not test association newsletters or the chatbot on your approved list, and we make no claims about current products. Applying the finding to a member research roundup is our reading. The Invented Ostrevale Society of Respiratory Therapists is made up.

Sources (2)