The Half That Cannot Be Automated

Summary

Validation is the standing tax on regulated software, and the usual response to a vendor claiming to reduce it is justified scepticism. The claim is nonetheless partly true, and the useful move is to say precisely which part. Validation cost has two halves that behave differently. Evidence generation — assembling the proof that a system does what it is specified to do, that its controls exist, that its records are attributable and contemporaneous — is largely a documentation exercise performed after the fact, and it is substantially a design problem. A system whose integrity properties are structural produces most of that evidence as a by-product of running. Evidence assessment — deciding how much rigour this system warrants, judging whether the evidence is adequate, approving the result, and carrying named accountability for it — is irreducibly human, is required to be, and does not shrink. Regulators have made this split easier to exploit: GAMP 5's second edition asks for demonstrated thinking rather than document volume, and the FDA's computer software assurance guidance concentrates effort where risk actually sits. Both reward a system whose design makes assessment cheap. The practical consequence is that "validation effort" is no longer one number to be discounted. It is two, and only one of them is negotiable — which is exactly the question to put to any vendor making the claim.

1. The two halves

Take a validation package apart and the work sorts into two kinds.

Generation. Writing the specification. Producing installation, operational, and performance qualification protocols. Executing them. Capturing screenshots and logs. Building a traceability matrix from requirement to test to result. Assembling the evidence that the audit trail works, that signatures bind, that access controls hold.

Assessment. Deciding the categorisation and the level of rigour warranted. Judging whether the evidence produced is adequate for the risk. Reviewing deviations and deciding what they mean. Approving. Signing. Being the named person accountable if a regulator disagrees.

Most conversations about reducing validation cost fail to make this distinction, and then argue past each other. The vendor is usually talking about generation. The quality function hears a claim about assessment, correctly identifies it as impossible, and discounts the whole thing.

The split matters because the two halves respond to completely different interventions. Generation responds to architecture. Assessment responds to nothing except experience and time — and it is not supposed to.

2. Why generation is a design problem

Most validation evidence is a retrospective account of a property the system either has or does not have.

Consider what qualifying an audit trail actually involves. A test is designed to demonstrate that changes are recorded with author, timestamp, and prior value. The test is executed. The output is captured. A document is written asserting that the requirement is met, referencing the evidence.

Now notice: the system was either recording those changes correctly or it was not. The validation activity did not create the property. It produced a description of the property, in a form a reviewer can assess.

That description is expensive because it is assembled by hand, after the fact, from a system that was not designed to describe itself. Where a system's integrity properties are structural — where the record is created by the act rather than reconstructed afterwards, where corrections append rather than overwrite, where the configuration governing behaviour is a readable versioned artefact — then most of the description already exists as a normal output.

Three consequences follow, and each is checkable.

Configuration review becomes evidence. If the rules governing a workflow are versioned readable files, the quality function reviews the actual configuration rather than a document describing it. The review is the evidence. There is no translation step, and therefore no gap between what was reviewed and what is running.

Traceability is a query, not a construction. Where requirements, configuration, and tests reference each other structurally, the traceability matrix is generated. Where they do not, someone builds it in a spreadsheet and rebuilds it at every change.

Change impact is computed, not estimated. The question "what does this release affect, and what needs retesting" has a determinate answer when configuration is declarative and versioned. Where it does not, the answer is a judgement call that defaults to retesting more than necessary.

Ask to see: the difference between two releases, expressed as a document a quality reviewer could sign — and then ask what else would need producing before that review could happen.

3. Why assessment is not

Four things in the assessment half are fixed, and it is worth being explicit about why, because the reasons differ.

Risk judgement is a professional determination. How much rigour a system warrants depends on what it affects, what fails if it is wrong, and what else catches the failure. GAMP 5's second edition formalises this as structured expert judgement by qualified people.¹ That is not a step waiting to be automated; it is the guidance asking for a human opinion by design.

Adequacy is contextual. Whether evidence suffices depends on the risk, the organisation's tolerance, prior findings, and the current regulatory climate. The same package can be adequate in one context and not another.

Approval carries personal accountability. Regulated change control requires a named human to accept responsibility. This is a legal property, not a workflow step, and no system property changes it.

Inspection response requires someone who can explain. When an inspector asks why a decision was made, the answer has to come from someone who made it and understood the reasoning.

Nothing in the architecture argument touches any of these. A vendor claiming to reduce total validation effort by some large percentage is either measuring only generation, or is describing a reduction that will not survive contact with a quality function.

4. Why the regime now rewards this

The split has always existed. What changed is which half regulators weight.

GAMP 5's second edition requires qualified experts to judge how much rigour a system actually needs, states that the lifecycle is not inherently linear, and supports iterative and incremental approaches.¹ The FDA's final guidance on computer software assurance takes a risk-based approach, concentrating effort where patient safety and product quality actually depend on it.² That guidance governs devices rather than drug manufacturing, but the direction is not ambiguous.

Read together, they move the centre of gravity from volume to reasoning. That is not a relaxation — a system whose design cannot be explained fares worse under this regime than under a document-counting one.

The commercial implication is the significant part. Validation effort is becoming a function of design clarity rather than a flat tax favouring the incumbent. Where revalidation cost was the moat protecting installed systems, a regime that rewards clear design erodes it. The vendors most exposed are those whose real advantage was the sunk cost of having been validated once.

5. What to actually ask a vendor

Six questions. Each has a wrong answer that is easy to spot.

"Which half are you reducing?" The answer should be generation, immediately and without hedging. A vendor who claims to reduce assessment does not understand the regime, or is hoping you do not.

"What evidence does the system produce without being asked?" Look for: audit records, signature records, configuration history, change manifests, access history — available continuously, without a support request. The follow-up matters more than the answer: what would we still have to produce by hand?

"Show me a configuration change as a reviewable document." If the answer requires a services engagement or a database export, configuration review cannot be the evidence and the generation saving is smaller than claimed.

"How is the traceability matrix produced?" Generated, or built? If built, it is rebuilt at every change, and that cost recurs forever.

"What happens to our validated state when you ship a release?" The good answer includes impact assessment, regression evidence, and updated traceability as a standard deliverable — not a quoted service. The revealing part is whether it comes by tier or by request.

"What did validation cost your last three customers, and what was the split?" Most vendors cannot answer. The ones who can are worth listening to, and the split is more informative than the total.

6. Scope of the claim

Where the claim applies, and what it costs.

The two halves are cleaner in this paper than in practice. Generation and assessment interleave — a reviewer reading evidence often identifies a gap, which generates more evidence. Treating them as separable is a simplification, and organisations with mature processes may find the boundary less crisp.

Regulatory direction is not regulatory permission. GAMP 5 and the CSA guidance point toward risk-based effort. A specific inspector, in a specific inspection, may still expect volume. Designing on the assumption that the direction has already arrived everywhere is a risk worth planning for.

7. The reframe

The useful question is not "how much validation effort will this save."

It is: which of our validation cost is describing the system to ourselves, and which is deciding whether we accept it?

The first is a design problem and it is negotiable. The second is the job, it is required to be the job, and a vendor offering to reduce it is offering something you should decline.

Appendix — Sorting a validation package

Take the last validation you completed and place each activity.

The ratio in your own package is the finding. A vendor's percentage claim can then be checked against it, rather than believed.

Sections 2 and 3 argue from mechanism. The claim that structural design produces "most" generation evidence is unquantified. No claim is made here about any specific vendor's validation footprint, including our own; §5 is written so a reader can test the claim against any vendor including us.