Evidence Chain / Proof-of-Work
Link conclusions to observed evidence and record missing coverage.
Evaluation & feedback
Scoring Rubrics
Define anchored criteria and action thresholds before assigning a score.
Self-Critique
Check a draft against named failure modes before presenting it.
Dual-Model Adversarial Planning
Develop alternatives independently before cross-reviewing their trade-offs.
Evidence-First Review
Demonstrate a defect through its trigger, path and consequence.
Rule-Catalog Review (YAML)
Store review rules as versioned data with explicit evaluation and failure actions.
Rubric-Based Model Evaluation
Use a model evaluator with a fixed rubric and checkable evidence.
Reader Testing with Sub-Agent
Test whether a fresh reader can understand the document without the author's history.
Blind A/B Comparison
Compare outputs on matched tasks while hiding version identity from the evaluator.
QA Sub-Agent with Visual Verification
Render the artifact and inspect its visible output before delivery.
Plan-Mandated Defect Is Still a Finding
Report an implemented requirement that is itself defective without hiding the conflict.
Diff-Scoped Review with Named-Risk Escape Hatch
Start review from the diff and expand only for a named cross-cutting risk.
Reviewer Does Not Re-Run the Implementer's Tests
Reuse valid test evidence and run focused checks when a concrete doubt remains.
Reachability-Scoped Delta
Scope upgrade advice to behavior the current code can actually reach.
Tool-Ran Tri-State Provenance
Distinguish tool presence, feasible execution and observed execution in reports.
Differential Baseline-Equivalence Oracle
Compare baseline and candidate behavior on identical inputs, including failures.
No-Op Test (Behavior-vs-Default Pruning)
Remove instructions only after testing their effect against the model's baseline behavior.
Two-Axis Orthogonal Review (No Cross-Axis Reranking)
Evaluate specification fit and engineering standards as separate verdicts.
Absence-of-Seam as a Finding
Report an untestable behavior boundary as an architectural finding.
Static-Trace Verification When Execution Is Unsafe
Use static tracing for unsafe execution and state the runtime evidence it cannot supply.
Validate measured artifact constraints
Measure the produced artifact against its delivery constraints and apply targeted corrections.
Pilot a fixed extraction contract
Freeze extraction fields and validate a small sample before scaling.
Abuse cases beside intended use cases
Derive negative scenarios from each trust boundary before selecting controls.
Grade the declared artifact kind
Choose the grading evidence according to whether the promised artifact is execution or conversation.
Validate the consumer interpretation
A local lenient reader passing is not proof that the real consumer accepts the same artifact.
Bind grader results to the full expectation set
Validate grade identities and reconstruct totals from the declared expectation set.
Review changes to the quality bar itself
Review the validator as well as the implementation so green cannot be bought by weakening the contract.
Measured baseline ratchet
Adopt a measured nonregression direction when an arbitrary target would fail the inherited baseline.
Noise-aware intervention comparison
Compare one change under equivalent measurement conditions and require signal beyond normal variance.
Owned negative routing controls
A negative routing example must route to its intended owner, not merely avoid one wrong skill.
Ordered reconciliation of reviewer findings
Treat reviewer output as hypotheses and classify against the actual artifact before acting.
Task acceptance and standing readiness
Require the requested behavior and the standing project readiness bar separately.
Reuse verification until its assumptions change
Repeat a successful check only when relevant code or baseline assumptions have changed.
Verify the evidence channel itself
Check the telemetry path by generating a known signal and finding it through the intended channel.
Keep answer keys outside agent reach
Make evaluation answers structurally unavailable to the system being tested, including history and earlier-trial artifacts.
Keep attempts, grades and traces coherent
Distinguish a scorable result from an attempt failure and keep each result’s grade, trace and usage bound to the same invocation.
Separate build variance from scoring variance
When a prompt builds an artifact that is later evaluated, measure build variability separately from repeated scoring of one artifact.
Concrete, intent-aware verification prompts
Attach optional verification questions to actual claims and assumptions, and omit them when the user already requested that verification.
Regrade all variants after a criterion change
When a metric or grader changes, apply it consistently to stored outputs from every compared variant before recommending a winner.
Use fresh no-change controls after drift
When execution conditions change, compare a candidate against a contemporaneous unchanged control rather than only an old baseline number.
Grade the environment left by an agent
For an agent that changes an environment, score the resulting state and use transcripts for explicit process constraints.
Freeze comparison references across rounds
Compare iterative candidates with the same saved reference artifacts and retain older references when adding a harder comparison.
Record the origin of evaluation references
Record whether reference answers are independently verified or generated by a system under comparison.
Keep selection feedback out of revision prompts
Keep withheld-case results out of the prompt that proposes revisions, and name the resulting selection set accurately.
Confirm the selected configuration as a whole
After selecting interacting settings in separate stages, evaluate the final combination on fresh runs against criteria fixed before confirmation.
Classify the failing layer when optimization stalls
When repeated edits fail to move an evaluation, classify remaining failures by the layer that could explain them before adding more instructions.
Evaluate the real application entry point
Exercise the application path that assembles context, routes tools and handles retries, rather than recreating its final model request.
Trace invalid data back to its origin
Walk backward from a failing effect through callers and values to identify where the bad input originated.
Establish counter semantics before totals
Determine whether metrics are increments or cumulative observations before aggregating them.
Use an independent, mutation-sensitive oracle
Derive expected values independently and check that plausible wrong behavior would fail the test.
Successor screens require an action and carried state
Validate a visual sequence as a causal user journey, not just a screen count.
Remove presentation answer cues
Keep an assessment from leaking its answer through presentation differences.
Guardrail negative-control proof
Verify a guardrail rejects an intentional violation, then returns to clean success.
Explicit partial and invalid fixture intent
Distinguish incomplete valid fixtures from intentionally invalid input.
Mechanics to checks, judgment to review
Turn syntactic mistakes into deterministic checks and reserve prose rules for judgment.
Compare structural variants in shared context
Compare design alternatives inside the real host context and hold data constant.
Qualify observations by when they were captured
Distinguish historical evidence, unverified snapshots, current observations and unknown state.
Hard Blockers Override Aggregate Scores
Apply declared non-negotiable blockers before deriving a readiness decision from an aggregate quality score.
Conditioned Description Is Not Independent Validation
Label a judgment that was given the measured answer as conditioned description rather than independent corroboration.
Choose Metrics from Failure Costs
Choose evaluation metrics according to the decision and the cost of its failure modes before optimizing a score.
Measurement Unit and Scope Boundaries
State what a measurement actually counts before using it as evidence for a different resource or behavior.
Related-Input Invariant Oracle
Check defined relationships between transformed inputs and outputs when exact fixtures alone leave broad behavior unchecked.
Compare Candidate and Baseline on the Same Trial Panel
Use the same declared trials for candidate and baseline and retain each paired result before applying an aggregate gate.
Propagate Failed Prerequisites
Revoke downstream passes that depend on a prerequisite which failed its own validation.
Retry Success vs Repeated Reliability
Distinguish succeeding once with retries from succeeding on every repeated trial.
Aggregate and Slice Evaluation Gates
Evaluate important task slices beside aggregate metrics using gates declared before inspecting the result.
Separate behavior success from intervention benefit
Report whether the requested behavior works separately from whether an added instruction improves it.
Comparable-coverage trend reporting
Compare result trends only when the covered checks and measurement conditions match.
Keep claims within the observed channel
Preserve a finding's evidence limits in every summary and proposed regression assertion.
Independent completion witness
Separate externally witnessed behavior from completion claims made by the code being evaluated.
Review both overlapping completion orders
Test a shared-state invariant across competing asynchronous operations, rather than reviewing each operation alone.
Persona-grounded peer benchmarking
Define the persona and journey boundaries before comparing steps and evidence; retain estimates, unknowns and pending target choices.
Predeclare trials and verdict rules
Fix a stochastic evaluation panel before seeing outcomes and retain every planned attempt.