Article
BeginnerWhen AI Helps Decide Something About a Member, a Person Has to Be Able to Change the Answer
Scholarship rankings, application flags and audit picks are decisions about people. The federal government's own AI rules for its agencies, issued April 3, 2025, offer a usable model: decide which uses are high stakes, put a person with real authority in front of each one, and give the member a way to ask for review.
AI Literacy Governance Membership Member Services
Picture the chair of a scholarship committee in early spring. There are 140 applications this year, five volunteer readers, and a deadline. The association’s grants software has a new feature: it will read every application and rank them. The chair turns it on. The committee agrees to read the top twenty closely and skim the rest. Three weeks later, the awards go out, and nobody can quite say whether application 87 was weak or whether the tool simply misread it.
This is a teaching example, not a case study. But versions of it are showing up across association work. Membership systems offer to flag incomplete or likely ineligible applications. Learning platforms offer to pick members for a continuing education audit. Community platforms offer to hide posts they judge to break the code of conduct. Award programs get shortlists. Each of these is a decision about a person, and each comes with a promise that a human stays involved.
The harder question is what that involvement has to look like for it to mean anything. One useful model comes from an unexpected place: the rules the federal government wrote for its own agencies.
A federal definition of high stakes
In a memorandum issued April 3, 2025, M-25-21, the Office of Management and Budget set out how federal agencies should govern the AI they use. It binds federal agencies, not associations. But it has to answer the same question an association faces, which is when an AI tool’s role in a decision is serious enough to need safeguards, and its answer is clear enough to borrow.
The memo defines “High-Impact AI: AI with an output that serves as a principal basis for decisions or actions with legal, material, binding, or significant effect on:” a list of areas. One item on that list is “an individual or entity’s access to education, housing, insurance, credit, employment, and other programs”.
Read that with an association’s programs in mind. A scholarship is access to money for education. Membership is access to a program, and often to a credential, a directory listing or a job board. An award shapes a career. If the AI’s ranking or flag is what the decision mostly rests on, the memo’s logic treats it as high stakes.
The memo also closes the most common escape hatch: “A high-impact determination is possible whether there is or is not human oversight for the decision or action.” In other words, having a person somewhere in the process does not by itself make a use low stakes. The scholarship committee that reads only the top twenty still has a human in the loop. The ranking is still the principal basis for who never gets read.
What the federal model asks for
For high-impact uses, the memo sets minimum practices. Several translate well to a membership office or a volunteer committee.
The first is oversight with teeth: “Agencies must ensure human oversight, intervention, and accountability suitable for high-impact use cases.” Intervention is the important word. A reviewer who can see the tool’s answer but cannot change it is watching, not overseeing.
The second is checking the tool over time. Agencies “must conduct testing and periodic human review of AI use cases, where feasible,” which for an association could be as simple as a committee comparing a sample of the tool’s rankings against its own reading each cycle, and noting where they differ.
The third is a path back for the person affected: “Agencies must ensure that individuals affected by AI-enabled decisions have access to a timely human review and a chance to appeal any negative impacts, when appropriate.” For a member who was declined, audited or passed over, that means knowing they can ask a person to look again, and getting an answer while it still matters.
The memo is practical about how to build that path. “If an agency already has an appeals or human review process in place, such as appeals of adverse actions, it may extend or adapt that process to cover decisions made with AI, consistent with applicable law.” Most associations already have something like this: a membership appeal, a grievance procedure for credentialing, a way to contest a CE audit. The work is to say plainly that it covers decisions where AI played a part.
It also asks that the path be easy to use: “Any remedy process should be designed to avoid placing unnecessary burdens on the individual”. A review that requires a formal letter to the board within ten days is a review few members will use.
Translating it for a small staff
None of this needs a new department. For most associations it comes down to three pieces of work.
Name the decisions. Make a short list of every place an AI tool scores, ranks or flags a member, applicant or volunteer. Include vendor features that were switched on by default. For each, note what happens next: does a person act on the output, or does the system act on its own?
Name the reviewer, and give them authority. For each decision that affects access to membership, money, a credential, an award or participation, write down who reviews it, that they review every case before it takes effect, and that they can change the outcome. If the committee reads only the top of a ranked list, the rest of the list has no reviewer.
Write the review path into the program rules. Add a sentence to the scholarship, membership or credentialing rules saying that applicants can ask for a person to review any decision made with help from an AI tool, and how. Point it at the appeal process you already have.
Back to the scholarship committee. Under this model, the chair would decide before turning the feature on whether the ranking would be the principal basis for anything. If the committee still wants the ranking, every application gets a human read, the ranking is one input among several, and the program rules tell applicants how to ask for a second look. If that is too much reading for five volunteers, the honest answer may be that the tool should not be ranking people this year.
The same thinking applies to hiring; our article on AI résumé screening covers that case and its own rules.
Checking that the review is real
Writing “a person reviews it” into a policy is the easy part. The harder part is checking, decision by decision, that the person reviews every case, does so before it takes effect, can change the result, sees the underlying record, and leaves a note of what they did.
Our tutorial on checking that a person really reviews every AI-assisted decision about a member walks through that check with a working lab, using a set of invented decisions at an invented association. It borrows its questions from draft UK regulator guidance and adds the federal idea that the member must be able to ask for a review.
An AI ranking can save a committee real time. It can also quietly become the decision. The difference is whether a person with authority stands between the tool’s answer and the member, and whether the member knows how to reach that person.