Part 2: Creating the Structure for Responsible Scale
Launching an AI pilot is often the easiest part of transformation. A small team selects a use case, tests a model, demonstrates value, and generates excitement. The harder challenge begins afterward: deciding who owns the system, how risks will be managed, how decisions will be documented, and how the organization will continue learning as the technology changes.
This is particularly important in life sciences, where AI adoption must move alongside regulatory expectations, patient safety, data integrity, scientific rigor, and public trust.
The FDA’s evolving AI journey offers a useful lens for understanding this next stage. A recent BioPharma Dive article describes continued interest in AI across the agency, including the development of Elsa and plans to explore agentic AI for premarket reviews, inspections, and administrative work. It also notes that leadership transitions have raised questions about how agency-wide AI initiatives will be coordinated over time. Importantly, the article reports that AI remains a priority and that the potential uncertainty relates primarily to the structure and pace of internal implementation rather than a reversal of the agency’s direction.
For pharma organizations, the lesson is broader than any individual tool or leadership transition. Responsible AI cannot depend only on momentum, enthusiasm, or one executive sponsor. It needs an operating model capable of carrying the work forward.
AI governance must become institutional
Many AI programs begin because one leader sees the opportunity and is willing to sponsor experimentation. That sponsorship is valuable. It can unlock investment, bring teams together, and give employees permission to explore new ways of working.
But an AI strategy becomes vulnerable when its ownership, funding, standards, and priorities remain tied too closely to one person.
The BioPharma Dive article discusses questions around the future leadership and governance structure for FDA-wide AI efforts following several senior departures. It also raises the possibility that AI activity could become more distributed across individual divisions rather than continuing through a centralized model.
There is nothing inherently wrong with distributed innovation. Individual teams often understand their own workflows and risks better than a central function. The challenge arises when every group develops its own standards, vendors, validation methods, data controls, and definitions of acceptable use.
For pharma companies, a durable governance model should make several responsibilities explicit:
- Who approves AI use cases
- Who owns the business outcome
- Who evaluates legal, regulatory, privacy, and ethical risk
- Who validates the system before deployment
- Who monitors performance after launch
- Who decides when a model must be changed, restricted, or retired
- Who remains accountable when an AI-assisted output influences a decision
These responsibilities should remain clear even when leadership roles, business priorities, or technology providers change.
The strongest model is often neither completely centralized nor completely decentralized. A central body can define common standards, controls, and decision rights, while individual functions retain responsibility for use-case design, workflow integration, and business value.
Transparency is part of trust
The article also raises an important question about transparency. If regulators use AI to assist reviewers, what should sponsors and CROs understand about those tools and processes?
Tala Fakhouri, a former FDA AI policy official quoted in the article, argues that greater clarity could help industry prepare submissions in ways that support both human reviewers and AI assistants. That could include clearer data labels, stronger information structure, and better organization of supporting materials.
Transparency does not require disclosing every technical detail of an internal system. It does require enough clarity for stakeholders to understand how AI is being used, what role it plays, and where human accountability remains.
Pharma organizations should apply the same principle internally.
Employees need to know:
- When an output was created or assisted by AI
- Which sources the system was permitted to use
- Whether the result requires human review
- What limitations are known
- How concerns or errors should be reported
- Who is accountable for the final decision
The absence of this clarity can create two opposite problems. Some employees may trust an AI-generated answer too readily because it appears polished and authoritative. Others may avoid the tool entirely because they do not understand how it works or whether using it is permitted.
Good governance makes responsible use easier. It should not force employees to interpret a complex policy every time they open an AI tool.
Governance should match the level of risk
Not every AI application requires the same level of review.
A system summarizing internal meeting notes is different from one influencing clinical-trial eligibility, safety assessment, regulatory submissions, promotional review, or patient communication. Applying the same governance process to every use case can either create unnecessary bureaucracy or provide insufficient control.
A risk-based model allows organizations to match oversight to potential impact.
A lower-risk application may require:
- Approved source data
- Basic privacy and security review
- Clear user guidance
- Human verification before use
A higher-risk application may require:
- Formal validation
- Documented performance thresholds
- Bias and error testing
- Audit trails
- Independent review
- Ongoing monitoring
- Defined escalation and shutdown procedures
The question should not be simply, “Is this AI?” It should be, “What could happen if this system is wrong, incomplete, biased, unavailable, or used outside its intended purpose?”
That framing leads to more proportionate and practical governance.
Traditional guidance and AI speed must coexist
The article highlights a tension that affects both regulators and industry. Traditional guidance processes provide stability, consistency, and an opportunity for careful review. At the same time, AI tools and platforms can change significantly within the period required to produce formal policy.
Fakhouri notes that even relatively fast guidance may take about a year, which can feel exceptionally long in the current AI environment. She suggests that more agile policy development and more frequent communication between regulators and industry may be necessary.
The answer is not to abandon rigor. Formal regulatory processes exist to provide clarity, fairness, and confidence. But formal guidance does not need to be the only mechanism for learning.
It can be supported by:
- Public workshops
- Technical discussion papers
- Frequently updated questions and answers
- Controlled pilots and sandboxes
- Cross-industry working groups
- Shared validation principles
- Case-based examples
- Regular regulator-industry forums
These mechanisms can help organizations understand emerging expectations before every issue is settled through final guidance.
The same principle applies inside pharma companies. A formal enterprise AI policy may be necessary, but it cannot remain unchanged for several years while the technology, use cases, and risks evolve around it.
Organizations need both stable principles and faster learning cycles.
Stable principles might include human accountability, privacy, traceability, scientific integrity, and appropriate validation. The implementation guidance beneath those principles can be reviewed more frequently as new use cases emerge.
Communication is a governance mechanism
Governance is often discussed as a collection of committees, policies, approval forms, and technical controls. Those elements matter, but communication is equally important.
Regular dialogue between data science, technology, legal, compliance, regulatory, medical, clinical, commercial, and business leadership can surface risks earlier and reduce late-stage resistance.
Too often, a team builds a promising AI solution and brings control functions into the conversation only near deployment. At that point, the investment is already significant, expectations are high, and necessary changes can feel like barriers rather than responsible design.
A better approach is to involve the right stakeholders from the beginning.
That does not mean every function must approve every technical decision. It means the team should understand early:
- What decision or workflow the AI will support
- Which data will be used
- Who could be affected by the output
- What evidence will be required
- How success will be measured
- What level of human review is appropriate
- What risks need to be designed out before launch
This turns governance from a final checkpoint into part of the product-development process.
Measure more than adoption
Organizations often measure AI success through the number of users, prompts, pilots, or hours saved. Those measures are helpful, but they do not tell the full story.
Responsible scale also requires measures such as:
- Accuracy and reliability
- Frequency and type of errors
- Human override rates
- Quality of source attribution
- User confidence
- Time saved without loss of quality
- Compliance with intended use
- Business and patient impact
- Incidents, near misses, and corrective actions
An AI tool may be popular because it is convenient, but convenience alone does not demonstrate that it is safe, effective, or appropriate.
Governance should create a feedback loop in which performance data, user experience, and emerging risks shape the next version of the system.
The goal is not to eliminate uncertainty
AI governance is sometimes approached as an attempt to remove all uncertainty before a tool can be used. That is rarely possible. Models change, data changes, user behavior changes, and new applications appear faster than organizations can predict them.
The more realistic goal is to build an organization that can recognize uncertainty, make responsible decisions, document its reasoning, monitor outcomes, and adapt when evidence changes.
The FDA’s evolving AI journey illustrates the importance of balancing continuity with learning, centralized standards with local expertise, and formal guidance with more frequent communication.
For pharma, moving beyond the pilot requires several commitments:
- Make AI governance part of the institution, not one leader’s initiative
- Establish clear ownership and decision rights
- Apply oversight according to risk
- Communicate openly about how AI is used
- Preserve human accountability
- Create faster mechanisms for learning alongside formal policy
- Monitor outcomes after deployment, not only before approval
The organizations that scale AI responsibly will not be those that create the most rules or launch the most pilots. They will be those that build governance strong enough to protect trust and flexible enough to keep learning.
Source: Kelly Bilodeau, “The FDA was all in on AI. Will that change?”, BioPharma Dive, July 28, 2026.
If this area interests you further or are interested to be part of the 2026 Pharma CX Marketing Summit in October, please do get in touch with us.