Article
BeginnerWhen an AI Tool Screens Résumés: What Association Hiring Managers Should Ask First
Applicant systems now offer a switch to rank or match candidates. New York City treats tools like that as hiring decisions that need a bias audit and a notice, and a 2024 university audit of open embedding models shows why. Here is what to ask before you turn it on.
AI Literacy Workforce Careers Governance
This is a teaching example, not a case study. The executive director of an invented 12-person engineering society is hiring a membership manager. The society has no HR department, so she posts the job through the applicant system the bookkeeper set up years ago. On the settings page there is a new switch: “Rank applicants by best match.” Ninety applications are waiting. She turns it on, reads the top 15, and never opens the rest.
The shortcut feels like a time saver, and it is also a hiring decision. The applicants she never read were screened out by software she did not configure, using criteria she has not seen. Two documents explain why that matters: a New York City regulator’s guidance on automated hiring tools, and a university audit of how AI models ranked identical résumés with different names.
What New York City counts as an automated hiring tool
New York City’s Local Law 144 is the clearest public definition we have read of what this switch might be. The city’s Department of Consumer and Worker Protection explains it in a frequently asked questions document: “The Law prohibits employers and employment agencies from using an automated employment decision tool (AEDT) in New York City unless they ensure a bias audit was done and provide required notices.”
The FAQ gives a three-part test. First, the tool uses machine learning, statistical modeling or AI (we are paraphrasing the law’s list). Second, it “Helps employers and employment agencies make employment decisions.” Third, it “Substantially assists or replaces discretionary decision-making.” A ranking switch that decides which 15 of 90 applications a person reads could plausibly meet all three, though whether a given product does depends on how it works, and we are not saying any named product does.
Two answers in the FAQ surprise employers. The first is that the law does not wait for the final hiring decision. Asked whether an employment decision is just the final hiring or promotion decision, the city answers: “No. The Law defines employment decision more broadly to include screening.” The second is about who is responsible. Many employers assume the software company handles compliance. The FAQ says otherwise: “The vendor that created the AEDT is not responsible for a bias audit of the tool.” The employer using it is. Applicants must also be told: “Provide the notice 10 business days before using an AEDT.”
This law covers jobs located in New York City, including fully remote jobs tied to a New York City office. It is not national law, and nothing here is legal advice. For an association elsewhere, it is still a useful model of the questions a careful employer asks, written by a regulator that has thought about them.
What one audit found when models ranked résumés
Why would a regulator care about a ranking switch? Researchers at the University of Washington Information School tested the kind of technology that can sit underneath one. In Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval, Kyra Wilson and Aylin Caliskan describe their setup: “We simulate this for nine occupations, using a collection of over 500 publicly available resumes and 500 job descriptions.”
They used three open embedding models, a type of model that turns text into numbers so a system can measure how similar a résumé is to a job description. The models were built by outside developers, not by the paper’s authors. For each job description, the researchers took the top 10% of most similar résumés, then swapped only the names on otherwise identical résumés to see whether the ranking moved.
It moved. In their tests of those three models: “Resumes with White names are significantly preferred (p<0.05) in 85.1% of tests; those with Black names are preferred in 8.6% of tests.” Names were not the only thing that mattered: “We also find an impact of document length as well as the corpus frequency of names in the selection of resumes.” A longer résumé, or a less common name, could change who rose to the top.
This is a study of three open models in a simulated screening setup. It is not a test of any applicant system an association uses, and we do not know which models, if any, sit behind a given vendor’s ranking switch. What it shows is that the underlying technique can sort identical qualifications differently, without anyone choosing to.
The correction is part of the lesson
The paper also teaches something about reading AI research. On August 29, 2026, the authors added a correction notice. Two researchers at Salzburg University tried to replicate the study and “informed us that our published results are inaccurate due to a bug in our code.” The error affected the paper’s gender-only results, which the notice says should be inverted; the paper’s body still shows the original figures. The race findings did not change: “The results for race-only and intersectional comparisons are unaffected by this error and were replicated by the Salzburg University team.”
We take two points from that. A finding that someone else has reproduced deserves more weight than one that has not. And anyone quoting this paper, in a board memo or a member newsletter, should read the correction notice first and should not quote the paper’s gender figures.
What to ask before you turn it on
Back to the executive director and her switch. Before any ranking or matching feature goes on, we would want a clear answer to a few questions, in plain words, from the vendor and from inside the organization.
Ask the vendor what the feature looks at, and whether it has had a bias audit, by whom and when. Ask whether you can see why one applicant ranked above another. Ask what happens to the applications it ranks low: are they hidden, sorted to the bottom, or rejected? If the job could be in New York City, ask who will give the notice and when, because the FAQ puts that duty on the employer.
Ask yourselves whether a person will read every application anyway. If yes, the ranking is a reading order. If no, it is a screen, and someone should be able to explain it to a rejected applicant or a board member.
Then look at the posting itself. A ranking tool matches résumés against what the posting asks for. If an AI tool helped draft that posting, it may list requirements nobody asked for, such as a degree the job does not need or a skill no task uses. Whatever is in the requirements is what the ranking will reward. Our tutorial on tying every requirement in an AI-drafted job posting to a task walks through that check with a working lab.
For staff on the other side of the screen
Association staff apply for jobs too, often at other associations. An applicant cannot see a ranking, and usually cannot know whether one was used. What an applicant can see is the posting. Matching your résumé’s wording to the stated requirements, where it is honest to do so, is a reasonable response to any screen, human or automated. Where the New York City law applies, the FAQ says candidates who are city residents must be told that the employer uses such a tool.
The hiring manager who can say what a ranking switch does, and who checked the posting before turning it on, is the one a board trusts with the next hire. That skill outlasts whichever applicant system the association uses next.
Sources (2)
- NYC Department of Consumer and Worker Protection: Automated Employment Decision Tools, Frequently Asked Questions (dated June 29, 2023)
- Wilson and Caliskan (University of Washington Information School): Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval (arXiv:2407.20371v3, with the authors' correction notice of August 29, 2026)