AssociationAI / AI Literacy
AssociationAI team Published

Article

Beginner

What Happens to the Words You Type Into an AI Tool: Logging, Training Use and Retention, Explained for Association Staff

A report commissioned by the European Data Protection Board traces where typed text can go in a hosted AI service, and the UK National Cyber Security Centre's 2023 view adds who can read it. Neither describes any named product. Here is what each step means for a membership desk.

AI Literacy Data Privacy Member Data Governance

Picture this illustrative case: the member services lead at a state trade association gets a long, angry email from a member about a disputed certification renewal. It names the member, their employer, a medical reason for the late filing and two staff. To save time, she pastes the whole thread into an AI chatbot and asks for a calm reply. The reply is good. The question nobody on her team can answer is where the pasted text went after she pressed Enter, who can see it now, and whether it is still there next year.

This is a teaching example, not a case study. The situation, though, is an everyday one for membership, certification and chapter staff, and answering it starts with a plain picture of how a hosted AI service handles input. Two public sources give that picture without describing any particular product: a report commissioned by the European Data Protection Board under its Support Pool of Experts programme, and a 2023 blog post from the UK National Cyber Security Centre. The first is one expert’s analysis, and its disclaimer says “The views expressed in the deliverables are those of their authors and they do not necessarily reflect the official position of the EDPB.” The second was written in March 2023 and says so. Neither tells you what the chatbot on your screen does today. What they give you is the set of questions worth asking.

The trip your text takes

The commissioned report describes the data flow in what it calls an “LLM as a Service” system, the kind of hosted tool most association staff use through a browser. It begins simply: “The process starts with the user submitting input, such as a query or command.” The input does not stay on your laptop. In the report’s words, “The input is sent through an interface or application managed by the provider (e.g., a webpage, app or a chatbot window embedded on a website).” Then “The API receives the input and routes it to the LLM model hosted on the provider’s infrastructure.” (report, page 28).

So the first fact for the member services lead is location. The member’s complaint, medical detail and all, was processed on equipment the association does not own or manage, under the provider’s rules rather than the association’s.

Where it can stay after the answer comes back

The reply appearing on screen is not necessarily the end of the trip. The report says “The provider may log the user input (query) along with the generated response to analyze the interaction and identify system errors or gaps in response quality.” It adds, about that same logged material, “The data could be also included in a training dataset to improve the model’s ability to handle similar queries in the future.” (page 28).

Notice the verbs: may log, could be included. The report is describing what a service can do, not what every service does. That is the honest position for any staff member too. You cannot know from the chat window alone whether a given tool keeps your text or trains on it. You can know that it is possible and that the answer lives in the provider’s own published terms.

The National Cyber Security Centre’s 2023 post approached the same question from a security angle. It said “An LLM does not (as of writing) automatically add information from queries to its model for others to query.” That is a useful correction to a common fear that whatever you type becomes instantly answerable for the next user. The same passage then says “However, the query will be visible to the organisation providing the LLM” and “Those queries are stored and will almost certainly be used for developing the LLM service or model at some point.” (NCSC, March 2023). It also notes “This could mean that the LLM provider (or its partners/contractors) are able to read queries, and may incorporate them in some way into future versions.”

Read together, the two sources describe a provider that can see the text, may log it, may keep it and could use it to improve a model, with people beyond the provider’s own staff possibly involved. For our member services lead, that means a member’s health information may now sit in a system where people outside the association could read it.

Logs are a second copy

Most staff picture the risk as the model itself remembering things. The commissioned report points first to something more ordinary: logs. It warns that “Logs from monitoring systems may retain personal data such as user interactions, creating risks of data leaks or misuse.” (page 25). In its risk table it adds “Providers can log user input queries and outputs for debugging or model improvement, potentially storing sensitive data” and “Logs containing user inputs and outputs could be accessed by unauthorized personnel or exploited in the event of a data breach.” (page 31).

The NCSC post makes the breach point in plainer terms: “queries stored online may be hacked, leaked, or more likely accidentally made publicly accessible”. A log is a copy of the conversation sitting somewhere else, and copies are what leak.

Many small pastes add up

One paste rarely looks dangerous. The trouble is volume. The report cautions “If logs are aggregated over time, they could form a comprehensive dataset that may reveal patterns about individuals, organizations, or other sensitive activities.” (page 31). The NCSC made the same point about accounts: “Also bear in mind aggregation of information across multiple queries using the same login.”

For an association, think of a shared staff login used for a year of renewal disputes, sponsor negotiations and board drafts. No single conversation reveals much. Taken together, they could sketch your members, your finances and your politics for anyone who can read the logs.

Training, memory and the long tail

On training, the report lists risks across the life of a model. For the training phase, its entry opens: “The model may inadvertently memorize sensitive data, leading to potential” (the sentence continues in the report). Later, during updates, “Using live user data for updates without proper consent or safeguards can violate privacy principles.” (page 25). The provider’s suppliers matter too: “If the provider relies on external cloud infrastructure or third-party tools for LLM processing, there’s an added risk of data exposure through those dependencies.” (page 31).

Then there is time. The report names “Lack of data retention policies: The provider could store the data indefinitely without having retention policies in place.” (page 31). And at the end of a model’s life, “Failure to properly erase personal data during decommissioning can lead to long-term privacy vulnerabilities.” (page 25). A paste made this autumn could, in the worst case the report describes, outlast the tool’s own product name.

What the member services lead does differently

None of this means association staff should stop using AI tools. It means the decision about a tool belongs before the paste, not after. The member services lead can still get help with a calm reply by describing the situation without the member’s name, employer or medical detail; our earlier post on what member data is safe to paste into AI covers that choice. For the tool itself, someone on staff should read what the provider says it keeps, for how long and whether it trains on input, and write it down with a date. Our same-day tutorial walks through that: find out what an AI tool keeps and trains on from its own terms.

A last caution about both sources. The commissioned report is written for a European data-protection audience and is not U.S. law or the Board’s own guidance. The NCSC post is more than three years old and was explicit that it described things as of writing. Use them to know which questions to ask, and use the provider’s current published terms for the answers.

Sources (2)