AssociationAI / AI Literacy
AssociationAI team Published

Article

Beginner

AI Proctoring Can Flag the Wrong Candidates on a Remote Certification Exam

In a 2022 study, the face check in one law-exam proctoring tool failed to verify the right test taker 11.47% of the time on average in the researchers' tests on image sets, covering only the part that runs on the candidate's computer.

AI Literacy Certification Governance Accessibility

In a USENIX Security 2022 study of the software used for law school and bar exams, the face-matching model in the suite with the largest market share failed to verify the right test taker 11.47% of the time on average, measured on image sets the researchers believed reflect a live exam. The researchers could examine only the part that runs on the candidate’s own computer, and they note the vendor may run further checks on its servers, according to the same paper. For a certification team that moved its exam online, that number is about who gets into the room, and who gets sent to review, as much as it is about a model score. The same study identified four suites used in 93% of U.S. law schools and 100% of remote state bar exams, which is the setting for those figures, not a claim about every certification exam. We read the finding as a warning for any desk that uses a face check as a gate to a high-stakes exam, and we say so as our reading, not as the researchers’ claim about associations.

What a failed face check does at an exam

That check decides whether a candidate can start the exam at all, and during the exam a turned head or a change in lighting can fail it and send the session to a person for review, according to the USENIX paper. The check can also be re-run during the exam, depending on the administrator’s settings, the authors write. Depending on how the flag is shown, the researchers warn, the reviewer may start from a presumption of guilt. They write that it is equally important to have “confidence that no individual who merits entrance into a profession has been blocked due to false cheating allegations.”

Three boxes in a row: a face check before the exam that can fail to verify the right candidate, checks during the exam that can raise flags, and a person who reviews those flags.

Who gets flagged for review

At the University of Louisville, researchers who examined one widely used proctoring product’s output for 357 students found that students with darker skin tones and Black students were significantly more likely to be marked for instructor review. In that study of one product, used with 357 students in four STEM courses in Fall 2020, darker-skinned women were 4.36 times more likely to be flagged overall than women with medium skin tones, according to the same paper. The USENIX team’s fairness tests also found that minority groups were flagged at higher rates, depending on the capture conditions.

When the flags do not track cheating

The Louisville team found no significant difference in observed cheating behavior by skin tone, race or sex. Of the 12 recordings in which the Louisville team saw possible cheating, the software had put 8 in its low priority group. The study did not collect information on whether instructors acted on those flags, the authors note.

The USENIX team also found that all the anti-cheating measures in the four suites could be bypassed, and that a candidate in full control of their own computer can always get around the monitoring given enough time.

Three moves before the next sitting

None of these moves asks you to take a flag at its word, or to assume a later version of a tool has fixed what the 2022 tests found.

  • Treat a flag as a reason for a person to look, never as a finding. Our November 7 walkthrough shows how to test whether that human review actually has authority to change the result. Where face recognition stays in use, the USENIX team recommends review by multiple diverse people.
  • Ask the vendor for face-check error rates broken out by group, on realistic images, since an overall rate hides the gaps, as the USENIX paper advises. The Louisville authors say vendors should be forthcoming about limitations. We think that request belongs in the contract renewal.
  • Keep a route that does not depend on the webcam. The USENIX team recommends a meaningful opt-out, with provided hardware or paper and live proctoring, and assessments designed to reduce the gain from cheating. Our companion tutorial, Match Each Testing Accommodation to Your Remote Exam’s Monitoring Rules, walks through accommodations on a remote exam.

The USENIX researchers’ recommendation is that bar associations and other licensing institutions investigate bias in the exam software they use. We think a certification exam committee can put the same question to its own vendor: what is the face-check error rate, broken out by group, on images that look like a live sitting? Both studies tested software as it stood in 2020 to 2022, in the researchers’ own tests, so a later version of a tool needs its own evidence.

Sources (2)