Output Evaluation & Validation

Output Evaluation and Validation is the heaviest domain on this exam at 21 percent — roughly one item in five, and the block that decides most passes. Nearly every scenario turns on the same beat: an answer arrives, it reads well, and you have to decide whether it is safe to act on before you send it, file it, or brief it to someone. Expect drafts that are fluent, confident, and quietly wrong.

The instinct being tested is narrow and learnable: judge output against a source, not against how plausible it sounds. Fluency is a property of the writing, not evidence for the claims inside it. A number, a date, a name, a quoted clause, a citation — each is a claim that must trace back to the document it came from. Tone, structure, and phrasing are not claims at all.

The idea that unlocks the domain is that verification is proportional, not maximal. Match the depth of checking to what breaks if the answer is wrong and whether it can be undone. Reversible internal work earns a skim; anything external, regulated, financial, or irreversible earns a named human reviewer before it leaves the building.

• Set the stakes before you read the draft. Ask what breaks if this is wrong and whether it can be undone. That answer sets how hard you check.

• Verify against a source, never against plausibility. Confident, well-organized prose is not evidence of anything.

• Check the load-bearing claims first: numbers, dates, names, quoted text, citations, policy or contract terms, anything a decision hangs on.

• Ask Claude to show where each claim came from — quote the passage, name the document, point to the section. Unsourced means unverified.

• Anything Claude could not have known from what you actually gave it is a claim to check, not a fact to accept.

• Tell Claude up front that "not in the source" is an acceptable answer. Give it a way to be uncertain and it will invent less.

• Spot-check long outputs by sampling: a fixed number of items spread through the document, every figure that feeds a total, plus the opening and closing sections.

• If a sample turns up one error, stop sampling and check the whole thing. Errors from a single cause repeat.

• Separate two questions: is it accurate, and is it fit for purpose. A true answer in the wrong scope, format, or tone still fails.

• Route to a named human before anything external, regulated, legal, financial, medical, or HR-related is used, and before anything irreversible.

• When the answer really matters, ask again from a clean start. A claim that changes between attempts was never solid.

• Record what you checked and against what, not just that you checked. When someone challenges the work later, the trail is the deliverable.

• The tempting wrong answer is usually "ask Claude if it's sure." Self-rated confidence is not verification, and it tends to mirror whatever you push back with.

• Do not accept a citation because it is formatted correctly. Real-looking references, page numbers, and titles are easy to fabricate.

• Do not read polish as accuracy. The best-written draft in front of you is not automatically the most reliable one.

• Do not check only the parts you understand. The unfamiliar section is exactly where you are least able to notice an error.

• Do not verify a summary against another Claude output. That tests consistency, not truth. Go back to the original document.

• Pasting a source into the prompt is not checking. Supplying the document and comparing the answer to it are two different acts.

• Do not re-read every long document line by line. Over-checking routine work burns the attention the genuinely high-stakes item needed.

• Do not under-check because the task felt easy. Stakes come from the consequence, not from how much effort the request took.

• A sign-off box is not a review. Approval by someone who did not read the output is a control that exists only on paper.

• Do not fix the error and move on. Ask why it appeared and whether the same fault runs through everything else in the output.

• Do not treat length as completeness. Padding is where unsupported claims hide most comfortably.

• Learn the confidently-wrong tells: very specific detail with no source, hedging that vanishes under pressure, and an answer that reshapes itself to agree with your objection.

VERIFY — the order to work through when an answer matters.

• V — Value at stake. What breaks if this is wrong, and can it be undone? Decide this before reading the draft, because it sets everything below.

• E — Extract the checkable claims. Numbers, dates, names, quotes, citations, obligations. Skip tone and phrasing; they are not claims.

• R — Request the source. Make Claude show where each claim came from: the passage, the document, the section.

• I — Inspect against the source. Compare the claim to the original material — not to how plausible it sounds, and not to a second Claude answer.

• F — Flag the unverifiable. Anything you cannot trace stays labeled as unconfirmed. It never gets quietly promoted to fact.

• Y — Yield to a human. Before anything external, regulated, or irreversible ships, a named person reads it and owns it.

Low value at V lets you run a light E and I and stop there. High value at V makes Y mandatory, not a courtesy.