Answers · By Hugh Donatello, Atlas Bio · Updated 2026-11-11

How to Run a First AI Agent Pilot in a Biotech: A 90-Day Plan

How do you run a first AI agent pilot in a biotech? Pick one non-GxP discovery task, such as triaging published literature, and name an owner. Scope the data the agent may read, run it in a sandbox on public, synthetic or copied data with no write access to systems of record, and write success and stop criteria before the run. Have a person review every output, record the agent in your AI inventory, and end with a go/no-go decision. Any later GxP use is a separate decision made through your quality system.

Download: 90-day AI agent pilot plan (Excel)
One row for each week-block in the plan on this page, plus the separate gate before any GxP use: what gets done in this page's words and, for each week-block, its written output; where this page cites a source for that step, the NIST AI RMF, EMA reflection paper or draft Annex 22 passage, or the OWASP or FDA document it names; and blank Owner, Date, Evidence and Decision columns. For the inventory entry it points to the AI agent inventory register template. No form or sign-up. Free to copy and use inside your organization. General guidance, not regulatory advice.
Printable PDF version

What a first pilot is for

A first AI agent pilot answers one narrow question: does this agent do this task well enough, under controls you can live with, to justify continuing? It is not a validation exercise and not a platform decision. Keeping it narrow is what lets you finish in about 90 days and make a clear decision at the end.

Start outside GxP. The European Medicines Agency's reflection paper on AI in the medicinal product lifecycle (September 2024) says the use of AI in drug discovery can be of low regulatory impact if non-optimal performance only affects the developer, but that if results contribute to the total body of evidence presented for regulatory review, the principles for non-clinical development should be followed. FDA's January 2025 draft guidance on AI to support regulatory decision-making for drugs and biologics states that it does not address AI used in drug discovery or for operational efficiencies that do not affect patient safety, drug quality or the reliability of study results. A discovery task whose output stays inside the research team is therefore the place with the least regulatory exposure to learn how agents behave on your work.

The plan below is a generic sequence built from public frameworks, mainly the voluntary NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1), with EU GMP texts borrowed as good practice where they help. It is not Atlas Bio's own method, and it is general information, not legal or regulatory advice. The week numbers are indicative; what matters is the order, and that each output exists before the next step starts.

The 90-day plan, week by week

Each period ends with something written down. If an output is missing, the next period waits for it rather than running in parallel. The success criteria in weeks 3 to 4 in particular must be fixed before the supervised run starts in week 7.

WeeksFocusWhat gets doneWritten output
1-2Choose the use case and ownerShortlist tasks; pick one non-GxP discovery task where a wrong output affects only internal work; name the owner and the scientists who will review outputs; describe the task, inputs, outputs and what the agent must not doIntended use statement with the reason the task is non-GxP
3Scope the dataList the data the agent may read; choose public, synthetic or copied data; review the model provider's terms on data retention and use with IT security and legal; exclude personal data unless your privacy or legal function has reviewed its useData scope note; provider terms reviewed
3-4Inventory entry and success criteriaCreate the agent's inventory entry; write the success criteria, how each will be measured, the current baseline, and the stop criteria that would end the pilot earlyInventory entry; success and stop criteria approved by the owner before any run
5-6Build and test in a sandboxSet up an isolated environment with no write access to production systems; log inputs, outputs and model version; test failure cases such as out-of-scope requests, instructions hidden in source documents and missing data; test the off switchSandbox test log; off switch tested
7-10Supervised runRun the agent on the real task inside the sandbox; a named scientist reviews every output before it is used; reviewers record accept, correct or reject and why; check the stop criteria weeklyReview log; issues list
11-12EvaluateScore the results against the criteria fixed in weeks 3 to 4 and the baseline; describe failure patterns; record any change to the criteria, with its reason, rather than rewriting themEvaluation report
13Go/no-go gateDecide to stop, extend the pilot, or continue in the same non-GxP scope; update the inventory entry; route any proposal for GxP use to a separate decisionSigned decision record; updated inventory entry

Weeks 1-2: pick a non-GxP discovery use case

Choose a task that is frequent enough to test properly, has an output a scientist can check, and where a wrong answer is caught before it matters. Examples that usually sit outside GxP include triaging or summarising published literature on a target (but not literature screening for adverse-event reporting, which is pharmacovigilance work), drafting a landscape of public clinical trial registrations, or proposing search queries for a database. Tasks to avoid for a first pilot are anything whose output enters a GxP record, a study report, a batch or release decision, a clinical document, a pharmacovigilance or safety assessment, or a regulatory submission, and anything that needs write access to a system of record.

Write the intended use down before you build anything. NIST AI RMF MAP 1.1 asks that intended purposes, context-specific laws, norms and expectations, and prospective settings in which the AI system will be deployed are understood and documented, and MAP 3.3 asks that the targeted application scope is specified and documented. A useful statement names the task, the inputs the agent may use, the outputs it produces, who receives them, and what it must not do. Add one line on why the task is non-GxP and what would change that, because the classification needs revisiting if the output later moves towards regulated work.

Name one owner in the research function the agent serves, and the scientists who will review its outputs. How roles divide across a wider governance programme is a separate question; a first pilot needs one accountable person and named reviewers.

Weeks 3-4: set success criteria before the run

Write down what success looks like, how it will be measured and what result would stop the pilot, and have the owner approve it before the supervised run starts. If the criteria are set after the results are in, the evaluation measures nothing. NIST AI RMF MEASURE 2.1 asks that test sets, metrics and details about the tools used during testing, evaluation, verification and validation are documented, and MEASURE 2.3 asks that performance or assurance criteria are measured and demonstrated for conditions similar to the deployment setting.

For a literature triage agent, criteria might cover how often reviewers accept an output without correction, whether any relevant paper in a reference set was missed, whether every cited source exists and says what the summary claims, and reviewer time per item compared with doing the task by hand. Choose your own thresholds; there is no standard number. Record the current baseline first, since a comparison needs one.

The European Commission's draft Annex 22 on AI, a GMP consultation text from July 2025 that is not in force and does not apply to generative AI or large language models, sets out a useful habit: acceptance criteria defined by a process subject matter expert and approved before acceptance testing starts, and criteria at least as high as the performance of the process the model replaces. A discovery pilot does not have to follow it, but the habit carries over. If you change a criterion during the pilot, record the change and the reason as a documented deviation from the plan.

Weeks 3-6: scope the data, then build in a sandbox

Decide what the agent may read before it reads anything. For a first pilot, prefer public data, synthetic data, or a copy of internal data held in the sandbox, so the pilot cannot change a live record, and do not let outputs made from copies of GxP records flow back into those records. List the data classes involved, such as public literature, proprietary results, sequences or personal data, and leave out anything the task does not need. Check the model provider's terms on retention and whether your inputs can be used for training before proprietary data is sent. OWASP's Top 10 for LLM Applications 2025 lists sensitive information disclosure (LLM02) among the main risks for LLM applications; our answer on how to secure AI agents in life science R&D covers the controls in more depth.

Build the pilot in an isolated sandbox: its own environment, its own credentials, no write access to production systems, and logging of the inputs, outputs and model version for every run. OWASP lists LLM06 Excessive Agency, damaging actions taken in response to unexpected, ambiguous or manipulated model output, usually rooted in more functionality, permissions or autonomy than the task needs, and source documents can carry text that tries to steer the agent (LLM01 Prompt Injection). Test those failure cases in weeks 5 to 6, before the supervised run: what the agent does when asked to act outside its scope, when a document contains instructions it should ignore, and when data are missing.

Plan the off switch at the same time. NIST AI RMF MANAGE 2.4 asks for mechanisms to be in place and applied, with responsibilities assigned and understood, to supersede, disengage or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use. In a pilot this can be simple: write down who can stop the agent and how its access is withdrawn, and test it once in weeks 5 to 6.

Weeks 7-10: human checkpoints, with the inventory entry made before the run

In a first pilot, a named scientist reviews every output before it is used, and records whether it was accepted, corrected or rejected and why. That log is the main evidence for the go/no-go decision. NIST AI RMF MAP 3.5 asks that processes for human oversight are defined, assessed and documented. Write down which outputs need review, who reviews them and how the review is recorded, rather than leaving it to habit. In its section on model deployment, the EMA reflection paper also asks that, for all models and especially those with no human in the loop, a risk management plan defines likely failure modes and how they will be monitored and corrected.

Record the agent in your AI inventory before the supervised run, not after. NIST AI RMF GOVERN 1.6 calls for mechanisms to inventory AI systems. The entry should carry at least the owner, the intended use, the data and systems the agent can reach, the model and version, its autonomy level, the actions that need a person's sign-off, the non-GxP classification with its reason, and the next review date. Our answer on what an AI agent inventory in pharma R&D should record lists those fields and maps each one to its source, with a free register template.

Week 13: the go/no-go gate, and a separate gate before any GxP use

End the pilot with a written decision. NIST AI RMF MANAGE 1.1 asks for a determination of whether the AI system achieves its intended purposes and stated objectives and whether its development or deployment should proceed. The options are to stop, to extend the pilot with a stated reason, or to continue in the same non-GxP scope. Base the decision on the criteria fixed in weeks 3 to 4, the review log and the issues list, and have the owner sign it. Stopping is a valid result: it is cheaper to learn that in a sandbox than in a regulated process. Update the inventory entry with the decision and the next review date.

A pass at this gate covers only the non-GxP use that was tested. Before the agent's output is used in a GxP record, process or decision, or in results that may contribute to the evidence presented for regulatory review, treat it as a new intended use: reclassify the inventory entry, record which regime applies (for example GMP, GLP, GCP or GVP) or, where none does, which regulatory expectations apply to that evidence, take it through your quality system's risk assessment and any computerised system validation the applicable regime requires, proportionate to risk, and get it approved under your quality system's procedures before use. Pilot results can inform that risk assessment. They count as validation evidence only where your validation procedures allow it, which usually requires that they were generated under conditions representative of the new intended use and documented to the standard those procedures require. Our answer on how to validate an AI agent used in a GxP process sets out those steps.

Two regulator texts show why this gate matters. As noted above, the EMA reflection paper says that where results contribute to the body of evidence presented for regulatory review, the principles for non-clinical development should be followed. The draft Annex 22, while still a consultation draft, says generative AI and large language models should not be used in critical GMP applications, those with direct impact on patient safety, product quality or data integrity, and that where they are used in non-critical GMP applications, qualified and trained personnel should always be responsible for ensuring the outputs are suitable for the intended use. Many agents are built on such models, so under the draft an LLM-based agent would be limited to non-critical GMP uses, with suitably qualified and trained staff responsible for checking each output, and any move into GMP work is a decision made through your quality system, not a continuation of the pilot.

A pilot is easier to scope once you can see how AI is already used. Atlas Bio's Secure AI for Life Sciences starts with an AI risk assessment: a map of how AI is used across your organisation today, including tools adopted without approval, where research data flows to external models and under which terms, gaps against the NIST AI RMF, and a prioritised plan. Status, October 2026. In use in Atlas Bio's own agent platform today: a separate verification step that checks agent work, and a hold that stops a task until a person decides. Built and tested on synthetic data, not yet used with a client: the AI risk assessment tooling (AI discovery, data-flow and vendor-terms mapping, and a NIST AI RMF gap report) and a record of what data an AI used, what it produced and who reviewed it. In development: a customer-controlled AI environment in your own Azure tenant, an AI gateway, agent permissions, evidence linkage and notifications. Each engagement states which controls are built. Decisions on GxP scope and validation stay with you, under your quality system.

Why start an AI agent pilot outside GxP?

Because a wrong output there affects your own research work, not a regulated record, a patient or a submission. The EMA reflection paper says AI in drug discovery can be of low regulatory impact if non-optimal performance only affects the developer. A non-GxP pilot lets you learn how the agent behaves on your work before any regulated record depends on it.

How long should a first AI agent pilot run?

Long enough to collect a review log you can judge it on, and short enough to force a decision. About 90 days, with a supervised run of roughly four weeks, suits many discovery tasks, but the order of steps matters more than the exact length.

Does a pilot need validation?

Not in the GxP sense. GxP computerised system validation requirements, such as EU GMP Annex 11, apply to systems used in regulated activities, not to a discovery task whose output stays inside the research team. It still needs written success criteria, testing and review. If the agent is later proposed for GxP use, or its results may contribute to evidence presented to a regulator, that use goes through your quality system as a new intended use.

Should the pilot agent have access to our ELN or LIMS?

Not for a first pilot. Give it public, synthetic or copied data in a sandbox with no write access to systems of record. Connecting an agent to a live ELN or LIMS is a separate decision with its own checks.

What if the pilot fails its success criteria?

Record the result, stop or extend the pilot with a stated reason, and update the inventory entry. Do not lower the criteria after the fact to get a pass; any change to the criteria should be recorded as a documented deviation from the plan, with its reason.

Can pilot results be reused if we later want GxP use?

They can inform the risk assessment for the GxP use. Whether any of the pilot's testing counts as validation evidence is decided under your quality system's validation procedures. It usually does not, because the pilot tested a different intended use on sandbox data.

  1. NIST AI 100-1: Artificial Intelligence Risk Management Framework (AI RMF 1.0)
  2. European Medicines Agency: Reflection paper on the use of artificial intelligence in the medicinal product lifecycle (September 2024)
  3. FDA draft guidance: Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products (January 2025)
  4. European Commission: Draft Annex 22 - Artificial Intelligence (consultation document, July 2025)
  5. European Commission: Stakeholders' consultation on EudraLex Volume 4 - Chapter 4, Annex 11 and new Annex 22 (closed 7 October 2025)
  6. OWASP Top 10 for LLM Applications 2025
  7. OWASP GenAI Security Project: LLM02:2025 Sensitive Information Disclosure
  8. OWASP GenAI Security Project: LLM06:2025 Excessive Agency
  9. OWASP GenAI Security Project: LLM01:2025 Prompt Injection

Talk to Atlas Bio.

Sponsors, researchers and clinicians: tell us the decision you are trying to make and we will show what the platform can and cannot answer.

Contact Us →