Adversarial and Hardening
How an agent leaks data, and how to give it less to leak
Scroll
How an agent leaks data, and how to give it less to leak
Exfiltration means getting data out of a system to someone who should not have it. An agent that can read private data and also takes in untrusted content can be turned against its own user this way. Instructions hidden in a web page or an email tell it to collect something sensitive and then send it somewhere. The sending can be surprisingly indirect. If the chat interface displays images automatically, the model can be led to write an image link whose address contains the data. The user's own app then requests that address, which delivers the data to whoever runs the server, without any tool call at all.
Three things together make this possible: access to private data, exposure to untrusted content, and a way to send information out. Remove any one and this kind of attack fails, so start by asking which of the three each feature really needs. A summarizer for public web pages does not need the user's inbox. An assistant that reads the inbox may not need to browse the web or display remote images.
Then give every tool the smallest set of permissions it can work with, a principle known as least privilege. Prefer read-only access, narrow scopes such as one folder or one project, and the user's own credentials over a shared admin key, so the agent can never reach more than the person using it. Restrict where data can go: keep a list of domains the agent is allowed to contact, and display images only from trusted hosts. Require confirmation for anything that sends, shares or deletes, and log every tool call with its arguments so a leak can be traced afterward.
Safety