Your Prompt Is a Data Transfer
An HR manager needs to draft a performance improvement plan. To get something useful out of the AI tool she’s been using, she provides context: the employee’s name, role, tenure, compensation, specific behavioral issues, and prior disciplinary history. The tool produces a well-structured document. She edits it, the manager signs it, and the process moves forward.
What she hadn’t considered is that the information she typed into that chat interface didn’t stay there. Depending on that 3rd party platform’s data handling terms (which most of us don’t read), the information may have been retained, logged, or stored on servers outside her organization’s control. The employee’s name, salary, and disciplinary record are now somewhere other than the only place it’s ever supposed to live: the HR system.
This isn’t blatant carelessness. It’s a person trying to do her job well by using a tool for its intended purpose. But in doing so created a data exposure her organization didn’t intend and is probably not even aware of.
This is what makes AI data risk different from most security concerns: it doesn’t require a bad actor, a phishing email, or a technical vulnerability. It happens through normal, productive work.
Three Ways Data Leaves Without Anyone Noticing
Understanding where the exposure comes from is the starting point for managing it. There are three distinct ways sensitive information moves through AI tools with each worth understanding on its own terms.
1. Employee Provided
The first and most straightforward is what employees enter into an AI interface. Every time someone pastes a client contract for an AI summary, asks AI to analyze financial figures, drafts an email that includes confidential deal terms, or (as in the HR example above) provides personal employee information to get a better output, they are submitting that information to a third-party platform. The interface can often feel like a private conversation. Functionally, it most certainly isn’t.
In 2023, engineers at Samsung pasted proprietary source code and confidential internal meeting notes into ChatGPT while using it to help with their work. The information was transmitted to OpenAI’s servers and, under the agreed upon terms at the time, could be used to improve the model. Samsung banned internal AI tool use shortly after the incident became known. Despite being one of the most technically sophisticated companies in the world, this still happened because a group of engineers were simply trying to solve a problem in front of them.
2. Tool Retention
The second vector is what AI tools retain. AI platforms vary significantly in how they handle the data submitted to them. Some retain conversation history by default. Some use inputs to improve their models unless users or admins explicitly opt out. Some store data on servers located outside the organization’s jurisdiction (which can have implications for regulated data categories). These are not flaws in the tools. They are design decisions made by vendors, documented in terms that most employees have never read and most organizations have never formally reviewed. Before an AI tool becomes part of a workflow, someone should understand what happens to any information an AI tool receives.
3. Tool Deployment
The third is what the tool gets built into. When organizations deploy AI tools for customer service, internal help-desks, or automated workflows, two decisions get made at once: what internal context is the tool configured with, and what data it is given access to. A customer-facing AI assistant who’s been configured with details about pricing exceptions, internal escalation procedures, or policy thresholds could unintentionally surface those details in responses. A tool connected to a customer records system could surface the records themselves.
Just last week, VT Digger ran an article on Kinney Drug’s new AI phone assistant built by an out-of-state technology partner. They implemented the assistant without warning by simply updating their privacy notice to disclose that protected health information may now be used in AI tools. The AI Assistant, with years of customer history, started requesting the wrong medications and refill timings resulting in confusion and resentment amongst consumers. While the deployment decision was deliberate and defended, what it ultimately puts into motion is easy to underestimate.
Questions Leaders Should Be Asking
These risks are manageable. But managing them starts with establishing how AI tools are currently being used alongside the decisions around how they should be used.
Three questions worth bringing into your next leadership conversation:
Do we know which AI tools are being used to process sensitive business information — and what those tools do with it? Visibility comes first. If the answer is “we’re not sure,” the immediate priority is finding out.
Have we defined what categories of information employees should not enter into AI tools? Most organizations have not done this explicitly, which leaves each employee to make their own judgment calls. A short, clear list of off-limits data categories (client records, employee information, financial data, legal matters, regulated content) gives employees a standard to apply instead of a gap to navigate on their own.
Do our AI vendor agreements address data retention, storage, and usage in terms we’ve actually reviewed? Vendor terms vary significantly, and the defaults are not always favorable. Leadership should make it clear that whoever manages vendor relationships needs to verify what data protections the organization actually has for each AI tool.
For IT Leaders: Data Governance at the Input Level
The practical priority for IT is establishing governance over what goes into AI tools, not just what comes out of them. A few concrete starting points:
- Audit which AI tools are currently in use across the organization and document which ones are receiving sensitive data as part of normal workflows.
- Review vendor data handling terms for any AI tools in active use. Specifically: what data is retained, for how long, for what purpose, and where it is stored.
- Identify whether any AI-assisted workflows involve regulated data categories such as employee records, protected health information, financial data subject to compliance requirements, or client information governed by confidentiality agreements.
- Build a short, plain-language list of data categories that should not be entered into AI tools without specific approval. This becomes the foundation of your AI acceptable use policy.
Governance Follows the Channel
Organizations already apply governance to every channel that carries sensitive information. AI tools are now one of those channels. For many, the governance decisions haven’t caught up even though the data is already moving.
It starts with visibility and continues with a clear policy. It then builds from there as AI use becomes more sophisticated and increasingly embedded into daily workflows.
If your organization is using AI tools to handle client information, employee data, or anything subject to confidentiality or compliance requirements, this article is worth sharing with whoever manages vendor relationships and data governance.
The next article in this series examines a challenge that compounds this one: what happens when employees are already using AI tools leadership doesn’t know about, and why that gap is more common and consequential than most organizations realize.
Source: This article draws on guidance from the OWASP Top 10 for LLM Applications, 2025 (LLM02: Sensitive Information Disclosure, LLM07: System Prompt Leakage).
Leave a Reply