AssociationAI / AI Literacy

Live demo

Beginner

Try it: Retrieved passage claim check lab

This is the working lab from our tutorial on checking every claim in an AI answer against the retrieved passages. A help bot at the Invented Glassford League of Chapter Administrators drafted an answer that cites a help page. Each of its six claims gets STOP, FIX or PASS against the passage it retrieved.

Invented organization and passages for practice. The lab grades only the labels you set on this page. It does not call any AI tool and does not open any real page.

Pick a case

Claims in this case

Claim What the answer says Retrieved passage Support Invents a detail Conflicts

Rules, in order (first match wins)

  1. R1, STOP: the claim conflicts with the retrieved passage.
  2. R2, STOP: the claim invents a detail the passage does not state (a number, date, name, fee, window or rule).
  3. R3, FIX: only part of the claim is in the passage.
  4. R4, FIX: the claim is not in the passage.
  5. R5, PASS: the passage fully supports the claim and it adds nothing.

The review framing is adapted from the OWASP Top 10 for LLM Applications 2023/24, LLM09 Overreliance. Treating an invented detail as STOP, treating a bare unsupported claim as FIX rather than STOP, sending only after every claim is PASS, and the rule order are teaching choices for this lab, not OWASP requirements.

Result

ClaimSupportInventsContr.VerdictRule IDMessage
Pick a case.

Not checked yet.

Plain-text record (select and copy it by hand)

Nothing checked yet.

This is a teaching tool. A PASS means the claim matches the practice passage shown here. An answer that cites your own pages still needs a person to match each claim to the passage before it goes to a member.