Scoring Rubrics

These examples and illustrative results are independently authored teaching materials, not measured model results.

Use case

Review incident repair items. Teaching A says only “improve retries”; B says “Li fixes retries and verifies one retry in a disconnect test.” A vague quality score cannot identify what A lacks.

Mechanism

Before scoring, define task anchors and action thresholds:0 no action,1 action without owner or completion check,2 all three. Cite each field/gap; below 2 requires revision. Unreadable material is unscorable rather than 0. Score items before aggregating and do not let totals hide essential omissions. Calibrate reviewer differences with common examples.

Bad example

Score the report 1–10 on overall quality and pass it if it feels good, without explaining individual repairs.

Good example

Score A and B with 0/1/2 anchors. Action, owner and completion check must all exist for 2. Quote A’s missing owner/check and request completion; inspect B’s fields. Preserve unknowns without inventing owners or impact, and do not replace item revisions with an average.

Why the change matters

Anchors connect judgment to observable fields, while thresholds connect scores to action. A low score identifies a repairable gap instead of abstract quality.

Observable expectation

Teaching A=1 and B=2; A needs an owner and check. Removing B’s owner also yields 1. Have reviewers classify common examples and compare reasons; clarify disagreeing anchors and verify consistency on new examples.

Limits

This editorial rubric is not model accuracy or objective measurement. Complete fields do not prove effective repairs; technical verification remains separate. Do not expand scope merely for a score. Check inter-rater agreement and sample fit before quantitative comparisons.