How should AI agent governance work in pharma R&D?
AI agent governance in pharma R&D should match the stakes: discovery work needs data protection and traceable sources, while GxP work, including GLP nonclinical studies, also needs, where the agent creates or is relied on for GxP records, a validated configuration, change control, audit trails and sign-off by an accountable person. Atlas Bio Systems explains both, and notes the Pistoia Alliance's agentic-AI standard initiative, announced in September 2025.
Why agents change the security problem
A chat assistant answers questions. An agent acts: it retrieves documents, queries databases, calls tools and can send data or trigger changes in other systems. In a research organisation that means an agent may touch unpublished results, assay data, regulatory drafts or partner material - the assets a company most needs to protect.
The OWASP GenAI Security Project's Top 10 for Agentic Applications ranks agent goal hijack - an attacker steering an agent away from its task - as the first risk. Its Top 10 for LLM Applications lists prompt injection first: instructions hidden in content the model reads, such as a document or web page, that the model then follows. An agent with broad access turns either failure into real actions.
The controls that matter most
The practical answer is the same discipline applied to any privileged account, adapted to how agents work.
- Identity and ownership: every agent is registered with a named human owner and a stated purpose.
- Least access and fewest tools: an explicit permission manifest says what the agent may read, call, spend and reach; everything else is denied.
- Untrusted input: documents, web pages and tool outputs the agent reads are treated as data, never as instructions.
- Human approval for high-impact actions: sending data outside the organisation, changing records or committing spend routes to a person.
- A kill switch: an agent can be suspended immediately without taking down the rest of the system.
- Complete logging: every tool call, input and decision is recorded so the action can be reconstructed later.
Discovery R&D vs GxP: what changes
Agent work in early research, such as literature review, target hypotheses, exploratory analysis and drafting, generally sits outside the GxP rules. There the controls above protect proprietary data and keep results trustworthy, and how far to formalise them is a business decision. FDA's January 2025 draft guidance on AI to support regulatory decision-making for drugs and biologics excludes the use of AI models in drug discovery. EMA's reflection paper says AI in drug discovery can be of low regulatory impact if non-optimal performance only affects the developer, but that if results contribute to the total body of evidence presented for regulatory review, the principles for non-clinical development should be followed.
What changes is the record. Once an agent creates, modifies or transmits records that FDA predicate rules require, or records submitted to FDA, 21 CFR Part 11 applies to those records, including its audit-trail requirements, read with FDA's 2003 Part 11 scope and application guidance. In the EU, GMP Annex 11 covers computerised systems used in GMP-regulated activities. The same kind of agent that drafts freely in a discovery workspace may need a validated configuration, change control and sign-off by the accountable person that the applicable rule names (for example a GLP study director, a clinical investigator, or the person the GMP procedure designates) when its output enters a GLP study report, a clinical record or a batch record.
A practical rule is to classify each agent by the most regulated record it can touch, and to keep discovery agents away from GxP systems unless they have been brought under those systems' controls.
| Question | Discovery R&D (non-GxP) | GxP work (GLP, GCP, GMP) |
|---|---|---|
| Which rules apply to the agent's records? | Company policy, contracts and data-protection law; no GxP record rules | Predicate rules plus 21 CFR Part 11 in the US; EU GMP Annex 11 for GMP computerised systems |
| Must the system be validated? | No regulatory requirement; testing is a quality and business decision | Yes, where the system creates, maintains or is relied on for GxP records or decisions; fitness for intended use must be shown, scaled to risk |
| Who approves or signs? | Whoever the company designates | A named, authorised person accountable under the applicable rule; an agent can prepare and route but does not sign |
| What does a model or prompt change need? | Whatever internal review the team sets | Change control under a defined procedure, with re-testing scaled to the change's impact on the validated state |
| What if discovery results go into a submission? | EMA: principles for non-clinical development should then be followed | Already in scope |
Frameworks to anchor the programme
The NIST AI Risk Management Framework (AI RMF 1.0) and its Generative AI Profile (NIST AI 600-1) give a structure for governing AI risk across the lifecycle - govern, map, measure and manage. Where an AI model produces information or data intended to support a regulatory decision about a drug's safety, effectiveness or quality, FDA's January 2025 draft guidance on AI to support regulatory decision-making for drugs and biologics describes a risk-based credibility assessment tied to the model's context of use; it does not address AI used in drug discovery, or AI used for operational efficiencies such as internal workflows or drafting a regulatory submission that do not impact patient safety, drug quality or the reliability of results from a nonclinical or clinical study. In Europe, the EMA's reflection paper on AI in the medicinal product lifecycle sets out EMA's considerations for AI used across that lifecycle.
At least one industry group is working on agent-specific standards for life sciences. On 4 September 2025 the Pistoia Alliance announced plans for an agentic AI initiative, with initial seed funding from Genentech, to develop a life-science and healthcare-specific agent-to-agent communication protocol and an AI agent standard. Its planned outputs are whitepapers, guidelines, reference implementations and scientific publications. The Alliance argues that full autonomy creates a 'black box' around AI that undermines trust, reproducibility and regulatory compliance. The announcement set out plans rather than a published standard, so check the Alliance's site for what has been published since.
Securing agents is easier when security is built into the environment they run in. Atlas Bio's Secure AI for Life Sciences starts with an AI risk assessment of current AI use and data exposure. Status, October 2026. In use in Atlas Bio's own agent platform today: a separate verification step that checks agent work, and a hold that stops a task until a person decides. Built and tested on synthetic data, not yet used with a client: the AI risk assessment tooling (AI discovery, data-flow and vendor-terms mapping, and a NIST AI RMF gap report) and a record of what data an AI used, what it produced and who reviewed it. In development: a customer-controlled AI environment in your own Azure tenant, an AI gateway, agent permissions, evidence linkage and notifications. Each engagement states which controls are built.
How should AI agent governance work in pharma R&D?
AI agent governance in pharma R&D should match the stakes: discovery work needs data protection and traceable sources, while GxP work, including GLP nonclinical studies, also needs, where the agent creates or is relied on for GxP records, a validated configuration, change control, audit trails and sign-off by an accountable person. Atlas Bio Systems explains both, and notes the Pistoia Alliance's agentic-AI standard initiative, announced in September 2025.
Does human oversight cancel out the efficiency gains of AI agents?
Not if it is designed in. Most agent actions run inside pre-approved boundaries without a person in the loop; only high-impact actions, defined in advance, need approval. The efficiency comes from agents operating at scale inside trusted boundaries, not from removing oversight.
What is the biggest risk with research AI agents?
An agent being steered off task - by prompt injection hidden in a document or by goal hijack - while it holds broad access to sensitive data. Limiting what each agent can reach limits what any successful attack can do.
Do we need a separate policy for agents?
Your AI policy should cover them explicitly: which agents are approved, who owns each one, what data each may touch, which actions need approval and how incidents are reported.
Do GxP rules apply to AI agents used in drug discovery?
Usually not to discovery work itself. GxP rules apply when an agent's output becomes a record that GLP, GCP or GMP rules require; the predicate rules and 21 CFR Part 11 then govern that record in the US, and EU GMP Annex 11 covers computerised systems used in GMP-regulated activities. Separately, Part 11 also covers electronic records submitted to FDA, and EMA expects AI results that contribute to the evidence presented for regulatory review to follow its principles for non-clinical development.
- OWASP GenAI Security Project: Top 10 for Agentic Applications for 2026
- OWASP GenAI Security Project: LLM01 Prompt Injection
- NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0)
- NIST AI 600-1: Generative Artificial Intelligence Profile
- FDA draft guidance: Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products
- EMA: Reflection paper on the use of artificial intelligence in the medicinal product lifecycle
- Pistoia Alliance: Pistoia Alliance unveils agentic AI initiative and seeks industry funding to drive safe adoption (4 September 2025)
- eCFR: 21 CFR Part 11 - Electronic Records; Electronic Signatures
- FDA guidance: Part 11, Electronic Records; Electronic Signatures - Scope and Application (September 2003)
- European Commission: EudraLex Volume 4, Annex 11 - Computerised Systems
- Does 21 CFR Part 11 Apply to AI Tools and Agents?
- What Is AI Agent Governance in Pharma R&D and GxP?
- How to Keep Proprietary Research Data Out of External AI Models
- AI Agent Audit Trails: 21 CFR Part 11 and Annex 11
- What Is Human-in-the-Loop AI in Drug Discovery?
- What Is an Agentic AI Pipeline in Biomedical Research?
Talk to Atlas Bio.
Sponsors, researchers and clinicians: tell us the decision you are trying to make and we will show what the platform can and cannot answer.
Contact Us →