For three editions now I have examined different aspects of the same system: an AI-enabled platform that incubates cells for advanced therapies, watches the culture as the process runs, and recommends the machine’s next move. The goal has not changed: fewer failed cultures, more therapies reaching patients, and lower cost.
I have also gone over how the system needs both computerized system validation (CSV) and context language validation (CLV), and how CLV follows a staged process much like CSV does: Define, Lock, Apply, Monitor, and Review.
In the Lock stage edition, the model failed because it was confident about something it had never seen, reading a new pattern as healthy instead of admitting it did not know. In the Apply stage edition, the authorization gate existed to catch the calls the model might not know it was getting wrong.
Both rested on the same assumption: that the system can recognize the limits of what it knows, and say so.
Recognizing those limits is not automatic. It must be designed in. That is the work of the Monitor stage of CLV, and it is the engine that makes the other stages safe.
The purpose of the Monitor stage is to reduce one of the model’s most dangerous risks: being confidently wrong about something it has not seen.
A closed-set model only knows the cell-culture states it was trained to classify: healthy, early stress, late stress, or abnormal. Show it a culture that fits none of those states, and it may still pick the most similar one. It may even report that pick with the same confidence it shows for a known state. Unless the system is built to reject, flag, or escalate the unfamiliar, it has no reliable way to say it has never seen this state before.
That is the trap.
A high confidence score does not mean the answer is right. The model shows its confidence immediately, but whether it was correct is often proven only later, once the culture’s true state is confirmed. Confidence is not correctness.
So the Monitor stage asks: is the live culture close enough to the approved reference bank to trust the model’s answer? When even the closest match in the bank is far away, the answer should be no, because the process is producing cell cultures the model was not trained to recognize.
The right response from the system in that moment is to flag, not to act as if the answer is trusted, and not to pretend to know. It should report its uncertainty and bring in a person to decide.
This changes what the model has to report: which state it picked, how sure it is, how close the case is to the approved reference bank, and whether the image sits near the edge of what the model has seen before.
How far is too far?
That is given by a boundary line, visualized as a closed loop drawn around the cases the system trusts: inside it, a culture is one it can act on; outside it, the culture is too far to act on without a person confirming first. How far a case sits from the line is measured by an approved similarity, distance, or out-of-distribution method. At first the line is kept tight, so the system asks for help more than it strictly needs to. Better to ask too often than to miss something real.
The line does not stay fixed. Run after run, review of the Monitor outputs shows which flags were confirmed as real and which were normal variation. With that, the line is adjusted, offline. Never on the live system. A new cutoff is proposed, and tested against held-back cases, edge cases, known failure patterns, and predefined acceptance criteria. Then it is approved through the proper change control, and locked as a new version.
Distance from the line is one signal: how far a case sits from the trusted set. The model’s own confidence is a second signal, but only when it is calibrated. Calibration helps make confidence meaningful within the validated operating range. A well-designed monitoring system should become less willing to trust the answer as a case approaches the line. Low confidence, when it is honest, is not a failure. It is the model signaling it is reaching the edge of what it was taught.
This is where the Monitor and Apply stages turn out to be two halves of the same safeguard. A human authorizing a call needs to see more than the model’s recommendation, or the approval is just a rubber stamp. To dissent, the human needs the evidence the Monitor stage creates.
The Monitor stage surfaces the doubt, the authorization gate decides what to do about it. Neither works alone: a signal that reaches no gate changes nothing, and a gate that weighs no signal is theater. Regulators do not accept theater. They expect objective evidence that the control worked as intended, that human review happened when required, and that the decision was recorded in a way that can stand up to inspection.
And Monitor never stops. A system that only checks its answers after the fact would see each flagged case as noise, just an error to filter out.
In fact, each flagged case is a question the bank could not yet answer, and therefore a candidate for the next version of what the model is told. That is the bridge to the last stage, Review, where those questions turn into controlled improvements.
But that is the next edition.
In a regulated process, being right most of the time is not enough. The system worth trusting is the one that can recognize the limits of its own knowledge and say so out loud, while there is still time for a person to act.
CSV asks whether the system performs as intended. CLV asks whether the context still supports that performance, and whether the system can tell when the model has reached the edge of that context.
The model’s most valuable output is not the answer. It is the moment it says it does not have one.
That’s the Minerva Way.
