A few weeks ago, in a workshop I was facilitating on compliance excellence in medical device development and manufacture, a Quality Systems Manager stated something I have heard, in some form or another, in almost every regulated AI conversation this year. After I indicated the next subject of the workshop was how to use AI compliantly to achieve excellence in change management, he said, “AI hallucinates.”
It was a reasonable statement, made in good faith, with thirty other people in the room waiting for my reaction. I am sure some were waiting for a wrong answer — a political one, a convenient one, a dismissal, an acceptance of defeat. I could feel the room running through each of them in the way a room does when a meeting has started to slip out of the facilitator’s hands. I did not have a clean way to address it. What I said was, “the context is the key to reduce hallucination.” It was not wrong, but it was not enough. I knew it before I finished the sentence, and so did the manager who had asked. And the audience.
I have been looking for the better answer since. Four weeks later I have it — from three independent routes, arriving at the same architecture that addresses the comment.
Route one — from the math.
Georgi Tancev, a risk engineer in pharmaceutical AI governance, published a paper in IEEE Transactions on Engineering Management arguing that three failure modes in AI large language models cannot be engineered away. The paper is the third in a four-paper transformation series he has been publishing since 2025 — an architecture for AI implementation in regulated pharma that includes operating-model redesign, knowledge architecture, and governance.
- The first failure is hallucination — this proves the manager right. The model confidently invents facts that are not in its sources. The mathematical result behind this is called diagonalization: for any finite test you can run on an LLM, there will always be inputs you did not test on which failure is guaranteed. No amount of additional testing, retraining, or scaling eliminates the existence of those failure cases.
- The second is what Tancev calls semantic quantifier failure — in simple words, this refers to the model’s inability to reliably handle statements that depend on the words all, every, none, no, always, never, each — the words logicians call universal quantifiers. The model fails at understanding what the quantifier word actually means in context. It cannot reliably grasp the logical weight of all-ness or none-ness — the requirement to check every case — and act on it. In pharma jargon: an LLM can sometimes tell you “some batch failed.” It cannot reliably tell you “all batches complied,” because checking all of something is a different mathematical operation than checking some.
- The third is output boundary impossibility — due to its own nature, the model cannot assign a true zero probability to any answer, which means even the wrong answer always has a non-zero chance of being produced. In simple words: there is no way to tell a language model that some answer is truly impossible. So you can make it very unlikely, but you cannot make it zero. Every time the model produces a word, every possible word in its vocabulary has some probability of being chosen — even the wrong one, even a dangerous one, even the one you explicitly told it not to say. The probability can be tiny. It can be a millionth of a percent. But it is never zero.
From the math, pharma AI governance cannot certify correctness. It must specify failure boundaries — what the system is permitted to do, what it is not, and what happens at the edge.
Route two — from the industry.
A day after Tancev’s preprint landed, Nuno Valério — Head of Innovation in R&D Quality — published Edition #6 of The Trust Architecture, the newsletter he writes from his experience with regulated organizations. In Edition #6, Valério reached the same conclusion as Tancev, from a different direction. He wrote: “The precedent already exists. Pharma has been doing failure-boundary specification on small molecules since 2009, under a different name: the ICH Q8 design space. We did not invent it for AI. We forgot we already knew how to do it.”
In other words: ICH Q8 indicates that design controls do not certify correctness. They inherently accept the natural variations in a pharmaceutical process. The solution is bounding the conditions under which the process behaves as intended. Behavior outside those bounds is a different question for a different control.
Based on the math and ICH Q8, a correct answer in the workshop might have been: “Yes, AI hallucinates — and it is the role of quality to ensure that when it does, the consequences stay inside what we can live with.” But that would have sounded defensive.
Route three — from the build.
The third route has been under construction at Minerva since its inception, in a different way. Minerva’s principle is that for maximum assurance, the instructions, requirements, risks and results should be seen — through visual-digital language. AI models are probabilistic. Their output cannot be trusted on inspection alone. Visual-digital language — what humans can interpret and act upon at a glance — is what makes the Trust Gate possible.
The Trust Gate exists because the failures Tancev names mathematically, and Valério names from the industry, are the failures Minerva has been engineering against — through visual-digital language.
The Trust Gate consists of two Virtual Verifications of the output from the model.
The first Virtual Verification is the human review gate, where AI output earns the status of knowledge after a qualified human signs it off, against a reference structure that makes the sign-off fast enough to be efficient and accurate. Without this gate, what the AI produced is information with probable failure, not verified knowledge — so the rest of the regulated process has nothing it can rely on.
The second Virtual Verification runs during execution, with dashboards, visual-digital readouts, electronic signals, and charts. The knowledge that passed the first gate can still degrade silently as conditions change. Without continuous monitoring, the validated state degrades and no one notices until a deviation takes place.
Two verifications by humans, both optimized visually and digitally for human comprehension and minimum risk of confusion. Both non-negotiable. The Trust Gate does not certify the model will not hallucinate. It is designed to contain, control, and — through continuous improvement — eliminate the consequences when the model, predictably, provides a wrong answer.
The convergence.
As with previous revolutionary changes in humanity, there is a lag in response when it comes to addressing the challenges and taking advantage of AI in regulated industries. We are not yet seeing a transformation in life sciences large enough to match it — the continuous improvement of our context language and information-sharing processes, the optimization of how humans with experience and knowledge participate in the work.
The frameworks that exist in the life sciences industry are responding to AI by adding controls inside the existing operating model. But that operating model needs to change. It needs to adapt to the probabilistic, non-zero-hallucination condition of AI. The redesign needed is an architecture where human knowledge is leveraged at maximum: routing and revision functions automated, judgment functions amplified, and the layer between revision and decision-making governed. The Trust Gate is not a control. It is the boundary.
Three independent routes. One conclusion. Stop trying to certify the model as perfect. Start specifying where it fails. Contain the consequences. Build the layer that makes the containment fast enough for regulated speed, and visible enough for regulated audit.
The Quality Systems Manager was not wrong. AI does hallucinate. And hallucination is not the problem — it is one of the predictable failures. The work is not eliminating hallucination. The work is understanding it, and containing it with the help of human knowledge and judgment.
Looking back at the workshop today, I can say I responded with the right instinct. The full answer is this: “A Trust Gate based on human knowledge and judgment is the key to contain AI hallucination.”
That’s the Minerva Way.
