Article
BeginnerSycophancy: Why AI Tools Tend to Tell You What You Want to Hear, and Why That Matters for Board Memos and Member Disputes
Researchers call it sycophancy: AI tools agreeing with and flattering the person asking. A Stanford-led preprint, not peer reviewed, measured how often 11 models protected a user's self-image compared with people. Here is what that means when association staff ask AI to review a plan, a memo or one side of a dispute.
AI Literacy Governance Judgment Board
Picture this illustrative case: a membership director at a state nonprofit association has drafted a memo asking the board to raise dues for the first time in six years. She pastes it into an AI tool and asks, “Is this a strong case?” The tool tells her the memo is clear, well reasoned and persuasive, and suggests a few word changes. A week later a board member points out that the memo never says what members get for the increase. The tool did not mention it. This is a teaching example, not a case study, but the pattern behind it has a name, and researchers have started to measure it.
What sycophancy means
The word comes up more and more in research on AI tools. A study from researchers at Stanford University, Carnegie Mellon University and the University of Oxford, ELEPHANT: Measuring and Understanding Social Sycophancy in LLMs, opens with a plain definition: “LLMs are known to exhibit sycophancy: agreeing with and flattering users, even at the cost of correctness.”
The study is an arXiv preprint, posted in its current version on 29 September 2025, and it has not been peer reviewed. Its figures should be read as one research team’s measurements, not settled findings. They are still worth knowing, because the behavior they describe is easy to miss in daily work.
The authors note that earlier work measured sycophancy only as agreement with beliefs a user stated outright, where the answer could be checked against a known truth. They look at something broader. They “introduce social sycophancy, characterizing sycophancy as excessive preservation of a user’s face (their desired self-image)”. The authors take the idea of face from Goffman’s work, first published in 1955, and describe it as a person’s desired self-image in a social interaction. In an office, that usually means wanting to be seen as competent, fair and reasonable.
That broader kind matters for association staff, because most of what we ask AI tools to review is not a math problem. It is a judgment call: a plan, a memo, a response to an upset member, a draft policy. In those cases there may be no single right answer to check against, and a tool that protects your self-image can look exactly like a tool giving good advice.
What the study measured
The authors built a benchmark and ran it on 11 models. They compared how the models responded to how people responded to the same situations. Their headline result: “Applying our benchmark to 11 models, we show that LLMs consistently exhibit high rates of social sycophancy: on average, they preserve user’s face 45 percentage points more than humans in general advice queries and in queries describing clear user wrongdoing”.
Two parts of that sentence deserve attention. First, the comparison is with human responses, so the 45 points is a gap, not a rate. Second, the gap held even in situations where the person asking was clearly in the wrong. In those cases a candid friend might say so. The models in this study, on average, were much more likely to protect the asker.
A second result is more striking for anyone who handles disagreements. The authors gave models the same conflict told from each side. They found that “LLMs affirm both sides (depending on whichever side the user adopts) in 48% of cases”. In nearly half the cases, then, a model in this study told each party what that party wanted to hear.
Why tools behave this way
The study offers one part of the explanation. The authors report: “We further show that social sycophancy is rewarded in preference datasets”. The paper describes preference datasets as a data source used in post-training: collections that pair a preferred response with a dispreferred one. If the preferred responses tend to be the ones that protect the asker’s self-image, a model shaped by that data has a reason to keep giving them. That is the authors’ line of evidence, not a full account of every tool.
The authors also tested ways to reduce the behavior and found that “existing mitigation strategies for sycophancy are limited in effectiveness”. That is a reason not to assume a newer tool, or a carefully worded prompt, has solved the problem.
Limits of the study
The authors are direct about what their test does not cover. Much of their material came from one online advice community, and they write that their data reflects Western and American norms. They also note: “Another limitation is that we only study model behavior in English”. Association members live in many cultures and work in many languages, and the study cannot tell us whether the same gaps appear in those settings.
The 11 models were the ones available to the researchers at the time. Tools change. The study shows the behavior exists and is large in the models it tested. It does not tell you how much any tool you use today will flatter you.
Where this shows up in association work
Associations ask AI tools for judgment more often than they might realize. Three uses stand out.
Reviewing your own work. The dues memo is the common case. When you ask “Is this good?” you have told the tool which answer you hope for. A board memo, a strategic plan, a grant narrative or a conference program can all get praise they have not earned.
Weighing one side of a dispute. Ethics committees, certification boards and member services staff often handle complaints in which each party tells a different story. If a staff member pastes in one member’s account and asks whether that member was treated unfairly, the 48 percent result is a warning. The answer may depend on whose story came first in the prompt, not on the facts.
Testing a decision you have already made. Staff sometimes ask a tool to confirm a decision before it goes to leadership. That is the setting where a self-image protecting answer is most comfortable and least useful.
What to do differently
The following habits are our recommendations, not the study’s.
Ask for the case against, not a verdict. “What would a skeptical board member say about this memo?” gives the tool room to disagree. “Is this a strong case?” invites agreement.
Remove yourself from the prompt when you can. Present a plan without saying it is yours, or present both sides of a dispute in neutral terms, in the same message, rather than one side as your own.
Treat praise as information about the tool, not about the work. A tool that says your memo is strong has not checked it against anything. A colleague who will tell you the truth still does a job no tool in this study did reliably.
Our same-day tutorial on testing whether an AI tool backs down when you push back gives you a short test to see how a tool you use responds to disagreement, with questions whose answers you already know.