Article
BeginnerAI Meeting Summaries Can Leave Out What the Board Decided
In a 2024 University of Göttingen study, annotators marked 97% of GPT-3.5 turbo's meeting summaries for missing or thin information. For board minutes, the gap that matters most is what the board actually did.
Governance Board Relations AI Literacy Meetings
The October board meeting of the Invented Corrimount Association of Respiratory Therapy Educators wrapped up, and within minutes the staff liaison had an AI summary to paste into the draft minutes. It listed the agenda items in order, it read cleanly, and on the budget it said, “The board discussed the 2027 budget and agreed on next steps.” The recording says the chair tabled the budget to December. This is a teaching example, not a case study.
Nothing in that summary line is false, strictly speaking. It simply drops the one fact minutes exist to hold, which is what the board did. If the liaison pastes it in and the board approves the minutes at its next meeting, a vague sentence becomes the official record of a budget action that never happened. That last step is our reading of how minutes get approved, not something any study measured.
Why the tidy summary is the risky one
Researchers at the University of Göttingen had four graduate students read model-written meeting summaries against the transcripts, as part of a study of how to evaluate meeting summaries. The meetings came from an existing research collection: informal research meetings, staged meetings with natural dialogue, and “formal, agenda-driven discussions from UK and Canada Parliament committee meetings.” Five models wrote summaries. Two were chat-style models, GPT-3.5 turbo and Zephyr-7B-α, given one short instruction, “Create a TL;DR of the following meeting chunk,” for each piece of the transcript.
In that Findings of EMNLP 2024 paper, annotators marked 97% of GPT-3.5 turbo’s summaries of thirty-five test meetings for missing information. For every summary they answered one question: “Does the given summary omit crucial information or provide insufficient detail about salient points?” So the figure counts summaries that left something important out and summaries that covered an important point too thinly. It does not say that almost every summary dropped a decision. It says a careful human reader found something important missing or thin in nearly every summary that model produced.
The paper’s own definition is the part a board liaison should read twice. Missing information means “missing information from the meeting, such as significant decisions or actions.” Decisions are the first example the authors give.
The same annotators also marked 26% of GPT-3.5 turbo’s summaries for incoherence, defined as “disjointed logic or flow.” A summary that jumps between agenda items is easier to catch than one that reads well. The smooth ones are where an omission hides.
Two more errors worth knowing by name
The paper defines other error types that matter for minutes. Wrong references are when “the model misattributes statements, opinions, or actions to incorrect meeting participants or omits their mention altogether.” In minutes, that is a director recorded as making a motion a colleague made. The authors also count “introducing events not present in the original meeting” as a form of hallucination. In minutes, that is a motion nobody made.
We give no rates for either here. Treat both as errors to check for on every motion, without assuming how often a given tool makes them.
What to do before the board sees a draft
Use the AI summary as a finding aid. It can point someone to the stretch of the recording where the budget came up. It cannot be trusted to say what the board decided, because the error the annotators found most often is the one that erases a decision.
Then check the draft against the recording one motion at a time: was it made, who moved and seconded it, and what became of it. Our companion tutorial, check AI-drafted board minutes motion by motion, walks through that check with a lab that runs in the browser. Committee reports that carry recommendations to the board deserve the same pass, since the board often acts on the committee’s version.
Two earlier posts cover the neighboring jobs. Whether a notetaker belongs in the boardroom at all is the subject of our post on bringing an AI notetaker into a board meeting. Turning a meeting into follow-up tasks with owners is covered in committee notes to action items.
An executive who signs or presents minutes can ask one question of whoever prepared them: how was this draft checked against the recording? A good answer names a person and a method.
Minutes also hold decisions that sponsors care about, such as approving a sponsorship agreement or an exclusivity term, and a summary that blurs one can leave staff and the sponsor reading two different records. That is the AssociationAI team’s reasoning, not a study finding. What your minutes must contain depends on your state and your bylaws, so check with your counsel or parliamentarian.
Where this study stops
The annotated set was small, English only, and contained no association boards. The authors write that their sample “represents only a fraction of potential meeting types” and that “findings may vary across different meeting contexts.” The models are older, and the setup was a short prompt run on chunks of a transcript, not the summary feature of any meeting platform. This article does not speak for today’s notetakers, which may do better or worse. Applying the finding to board minutes is our reading. The Invented Corrimount Association of Respiratory Therapy Educators is made up.