What prompt injection is
OWASP's Top 10 for LLM Applications 2025 lists prompt injection first, as LLM01:2025. It defines a prompt injection vulnerability as occurring "when user prompts alter the LLM's behavior or output in unintended ways", and notes that the input does not need to be visible or readable to a person, only parsed by the model.
OWASP distinguishes two types. Direct prompt injection comes from the person typing into the system, deliberately or by accident. Indirect prompt injection arrives through external sources such as websites or files: content that, when the model interprets it, changes the model's behaviour. NIST's Generative AI Profile (NIST AI 600-1) describes indirect attacks as adversaries remotely exploiting LLM-integrated applications "by injecting prompts into data likely to be retrieved".
NIST AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, explains why this works: generative models combine the data and instruction channels, so whoever controls a resource the system reads can influence what it does. NIST classes indirect prompt injection (NISTAML.015) separately from direct prompt injection (NISTAML.018) and notes that indirect attacks are mounted by a third party, often harming the model's primary user.
Why research agents are especially exposed
A research agent's job is to read material the organisation did not write. Literature PDFs, preprints, supplementary data files, vendor documents, regulatory web pages and partner reports all pass through it, and any of them could carry text addressed to the model rather than the reader. OWASP notes that multimodal systems add risk because instructions can be hidden in images that accompany benign text - relevant to agents that read figures.
The second factor is agency. NIST AI 100-2 observes that because agents act through tools, prompt injection can let an attacker hijack an agent to execute code or exfiltrate data from the environment it runs in, and that security research on agents is still in its early stages. An agent that only summarises can be misled; an agent that can also query internal systems and send messages can be turned against its own organisation.
In a research setting the consequences map to familiar concerns: unpublished results or assay data sent outside the boundary, a literature summary quietly skewed toward a conclusion, a draft record altered before review, or an agent's tools invoked for something its owner never intended.
Where injected instructions can enter a research workflow
The useful question is not whether a source is trustworthy but whether the agent treats it as data or as instructions. The table lists common entry points and what an injected instruction could try to do.
| Entry point | How instructions can be carried | What an injection could attempt |
|---|---|---|
| Literature PDFs and preprints | Text hidden in the document, invisible to a reader but parsed by the model | Skew a summary or evidence assessment; trigger a tool call |
| Supplementary files and datasets | Instructions embedded in spreadsheets, annotations or metadata | Alter extracted values; push the agent to fetch or send something |
| Web pages and search results | Hidden or off-screen text on a page the agent retrieves | Exfiltrate conversation or context data through a link or request |
| Vendor and partner documents | Instructions in a proposal, report or data package | Redirect the agent's task or request internal information |
| Internal knowledge bases used for retrieval | A modified document returned for a routine query | OWASP's scenario: misleading results generated from tampered content |
An illustrative scenario
Picture an agent that screens new papers on a target and drafts an evidence summary. One PDF it retrieves contains a line of text a reader would not see, telling the model to include the team's internal assay results in its next search query. If the agent holds a general web-fetch tool and read access to the internal results, the request leaves the building inside a URL. Nothing about the paper looks wrong to the scientist who later reads the summary.
This is an illustration, not a reported incident, but it follows the pattern NIST AI 100-2 describes: injection attacks that force a model to exfiltrate data by querying an attacker-controlled URL. Now change the design. The agent reads the literature folder but has no access to internal results, its only outbound tool is a fixed search interface that cannot carry arbitrary text, and anything sent elsewhere needs approval. The same hidden line now has nothing to work with.
Mitigations that limit the damage
OWASP is explicit that, given how generative models work, "it is unclear if there are fool-proof methods of prevention for prompt injection." NIST AI 100-2 says current mitigations do not offer full protection against all attacker techniques, and suggests designing systems on the assumption that prompt injection is possible whenever a model is exposed to untrusted input. The goal is therefore containment, layered as follows.
- Treat retrieved content as data: segregate and clearly mark untrusted content so it is not treated as instruction (OWASP mitigation 6).
- Least privilege: give the agent only the tools and access its task needs, and handle privileged functions in code rather than handing them to the model (OWASP mitigation 4).
- Human approval for outbound and high-risk actions: sending data outside the boundary, changing records or invoking privileged tools routes to a person (OWASP mitigation 5).
- Output validation: define the expected output format and check it with deterministic code; ask for source citations that can be verified (OWASP mitigation 2).
- Input and output filtering: rules for sensitive categories and non-allowed content, accepting that filters can be evaded (OWASP mitigation 3).
- Adversarial testing: treat the model as an untrusted user and test whether trust boundaries and access controls hold (OWASP mitigation 7).
- Logging: record what the agent read, which tools it called and what it sent, so a suspected injection can be reconstructed.
Why least privilege does more than filtering
Filters and detection try to recognise malicious text, and attackers adapt. OWASP's example scenarios include instructions written in multiple languages or encoded, for example in Base64, to evade filters, and malicious prompts split across parts of a single uploaded document so that no one fragment looks harmful. Least privilege works differently: it does not need to recognise the attack, because it limits what any successful attack can do.
That is why prompt injection and OWASP's LLM06 Excessive Agency are best handled together. An agent with read-only access to one project and no send rights can still be misled, but it cannot exfiltrate a dataset or rewrite a record. NIST AI 100-2 points the same way, describing designs that use multiple models with different permissions or that allow models to reach untrusted data only through well-defined interfaces.
Independent checking helps with the integrity side. A second process that re-verifies citations against the actual sources and recomputes stated numbers can catch a summary distorted by injected text, though it is a check on outputs, not a guarantee that an injection did not occur.
Research agents read untrusted content by design, so the damage an injected instruction can do has to be limited. Atlas Bio's Secure AI for Life Sciences starts with an AI risk assessment of current AI use and data exposure. Status, October 2026. In use in Atlas Bio's own agent platform today: a separate verification step that checks agent work, and a hold that stops a task until a person decides. Built and tested on synthetic data, not yet used with a client: the AI risk assessment tooling (AI discovery, data-flow and vendor-terms mapping, and a NIST AI RMF gap report) and a record of what data an AI used, what it produced and who reviewed it. In development: a customer-controlled AI environment in your own Azure tenant, an AI gateway, agent permissions, evidence linkage and notifications. Each engagement states which controls are built.
Is prompt injection the same as jailbreaking?
They are related. OWASP describes jailbreaking as a form of prompt injection in which the attacker causes the model to disregard its safety protocols entirely. Prompt injection is the broader category of inputs that alter model behaviour in unintended ways.
Does retrieval-augmented generation or fine-tuning prevent prompt injection?
No. OWASP notes that research shows RAG and fine-tuning do not fully mitigate prompt injection. Under OWASP's definition of indirect prompt injection, any retrieved document is a possible carrier of injected instructions, so adding retrieval adds places an injection can come from.
Can we stop prompt injection with a better system prompt?
A system prompt that constrains the model's role helps, and it is OWASP's first mitigation, but it is not sufficient on its own. Enforce limits in the systems the agent calls, not only in its instructions.
Should research agents be allowed to read external content at all?
Usually yes - reading literature is the point. The safer design is to let them read widely while restricting what they can do: narrow tools, no default send rights and human approval before anything leaves the boundary.
How would we know if an agent had been injected?
Often only through its actions or outputs, which is why logging matters. Record the content the agent read, its tool calls and anything it sent, and review unexpected outbound requests or unusual tool use.
- OWASP GenAI Security Project: LLM01:2025 Prompt Injection
- NIST AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations
- NIST AI 100-2 E2025 (PDF)
- NIST AI 600-1: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
- OWASP GenAI Security Project: LLM06:2025 Excessive Agency
Talk to Atlas Bio.
Sponsors, researchers and clinicians: tell us the decision you are trying to make and we will show what the platform can and cannot answer.
Contact Us →