AssociationAI / AI Literacy
AssociationAI team Published

Tutorial

Beginner

Write the Handoff Rules for Your Member Chatbot Before It Goes Live

Before a chatbot answers members on your website, write down who it hands off to, how fast a person replies, which topics it should never finish alone, how it says it is a bot, and when its content was last checked. Then test each setup against the rules.

Time needed: About 30 minutes for one chatbot

Before you start:

  • Access to your chatbot's settings, or the vendor contact who configured it
  • A list of the questions members bring to member service, such as renewal, billing and CE credit

AI Literacy Member Services Membership Governance

By the end of this walkthrough you will have a one-page handoff policy for your member chatbot and a verdict for each setup: launch it, fix it first, or keep it offline. Most bots arrive configured by a vendor, and nobody on staff has written down when the bot should stop and pass a member to a person.

Try it now: our chatbot handoff rule lab checks six invented chatbot setups against six handoff rules and gives each a STOP, FIX or PASS. It runs in your browser and checks only what you enter.

Where the rules come from

The Administrative Conference of the United States, which studies federal agencies’ procedures and recommends improvements, adopted Recommendation 2022-3 on chatbots and other tools that answer the public’s questions about the law. It is addressed to federal agencies, not associations, and it does not bind you. We use it as a model because its worry fits a member portal: “At the same time, the seemingly personalized nature of the guidance provided by such tools may lead users to unduly rely on it.”

Applying it to member billing, eligibility and complaints is our own adaptation, not something ACUS says, and so are the lab’s time limits.

Write the handoff policy

  1. List what the bot answers. One line per topic: renewal dates, event refunds, CE credit status.
  2. Name the person or queue behind each topic. ACUS says agencies “should design automated legal guidance tools to put users in contact with a human customer service representative to whom they can address questions in the event that a question is not answered by an automated legal guidance tool or if the users are having difficulty using the tools.” A shared inbox counts if someone owns it. A “contact us” link that reopens the chat does not.
  3. Set a reply time. The lab uses 48 hours as its limit. That number is our teaching choice; pick one your staff can keep.
  4. Mark the topics the bot should never finish alone. The recommendation reads: “When the underlying law is unclear or unsettled, or when the application of the law is especially fact-dependent, agencies should be transparent about the limitations of the advice the user is receiving.” A billing dispute or a question about whether someone qualifies for a credential depends on that member’s own record. Set those topics to say what the bot cannot decide and offer a person.
  5. Make the bot say it is a bot. ACUS: “When applicable, agencies should provide disclaimers that the automated legal guidance tool is not human.” Put it in the first message, not a footer.
  6. Give members a copy of the chat. The recommendation asks agencies to “allow users to obtain a written record of their communication with automated legal guidance tools and should include date and time stamps on the written record.”
  7. Date the content and invite corrections. Three lines apply. “Agencies should update the content of automated legal guidance tools to reflect legal developments or correct errors in a timely manner.” “Agencies should provide the date on which the tool was last updated.” And: “Agencies that use automated legal guidance tools should provide users the ability to offer feedback or report errors.” The lab flags content reviewed more than 180 days ago, another teaching choice.
  8. Run each setup through the lab. Clear every STOP before launch and schedule every FIX.

Worked example: six setups at one society

Renee Castillo, the invented member services director at the Tidewater Society of Practice Managers, lists six chatbot setups with Owen Pratt, the web manager. The lab checks the rules in order, and the first match decides.

Six invented chatbot setups: the renewal bot passes; the event bot and CE bot are marked STOP; the billing, credential and portal help bots are marked FIX.

  • A, renewal bot: named queue, 24 hours, every box yes, content reviewed 30 days ago. PASS (R7).
  • B, event bot: its contact link leads back to the bot, and it never says it is a bot. STOP on R1, because R1 is checked before R2.
  • C, CE bot: a named person behind it, but no notice that it is a bot. STOP (R2).
  • D, billing bot: billing disputes are not marked for a person. FIX (R3).
  • E, credential bot: a person answers in 49 hours. FIX (R4). Change it to 48 and it passes.
  • F, portal help bot: content last reviewed 181 days ago, and no copy of the chat. FIX on R5, which comes before R6. At 180 days the content check passes and R6 still asks for the copy.

For case D the lab says: “Mark billing disputes, eligibility and complaints as ‘say the limits and offer a person.’” The summary line reads “6 setups checked: 2 STOP, 3 FIX, 1 PASS.”

Check your result

  1. Every topic on the list has a named person or queue behind it.
  2. Someone has tried the path to a person and reached one.
  3. The first message says the member is talking to a bot.
  4. Billing disputes, eligibility questions and complaints go to a person.
  5. Members can get a dated copy of the chat.
  6. The content shows a review date and a way to report errors.

A PASS here means the setup meets these teaching rules; it is not a legal review. This is general information, not legal advice.

Where handoff rules usually break

A contact link that loops. The “talk to us” button opens the same bot again.

A queue with no owner. Handoffs land in an inbox nobody checks on Fridays.

Content that is never re-dated. Last spring’s refund policy keeps answering after the board changed it.

Sources (1)