P245 · Evaluation & feedback

Owned negative routing controls

A negative routing example must route to its intended owner, not merely avoid one wrong skill.

Editorially reviewed

These examples and illustrative results are independently authored teaching materials, not measured model results.

Use case

Test whether “write a regression test” incorrectly routes to code-review rather than test-driven-development. Review scoring 0 while every skill also scores 0 does not establish correct ownership.

Mechanism

Declare and validate the negative case’s owner. Require the wrong skill not to be a nonzero top match and the owner to match nonzero above it, recording ranks/tie rules. Add owner-positive and nearby ambiguous cases. Separate lexical runner judgments from actual host invocation, which still requires behavioral checks.

Bad example

Pass because code-review scores 0 for the regression prompt, even when the test skill also scores 0 and nothing owns the task.

Good example

Name test-driven-development as owner. Require a nonzero score above code-review; all-zero matches, missing owner or a nonzero wrong top candidate fail. Retain ranks and label lexical approximation, then separately test actual triggering.

Why the change matters

A negative case should verify intended ownership, not merely avoidance. An owner condition prevents vacuous no-match passes and exposes narrow descriptions or missing corpus entries.

Observable expectation

Teaching review=0/test=0 fails; review=0/test=2 with test ranked first satisfies the lexical contract. A misspelled owner is configuration failure. Actual host review invocation still fails behavior despite lexical success.

Limits

Scores are not invocation probabilities or proof of host routing. Define ties, multiple valid owners and abstention. Models, descriptions and available skills affect results; scores here are illustrative.

Sources and evidence

Read the editorial criteria