Capability Smoke-Test Over Presence-Check
Exercise a capability on a small real input before relying on its availability.
These examples and illustrative results are independently authored teaching materials, not measured model results.
Use case
An installed source analyzer may lack the project parser. Teaching input has one known function needed for definition lookup. Presence differs from effective capability.
Mechanism
Verify binary/version/read-only interface and exercise a small representative input. Inspect exit/diagnostics/identity, not just no exception. Classify missing parser/config/support and use documented fallback without unauthorized writes/installations.
Bad example
Command presence triggers full analysis; an empty success object proves language support without inspecting the function.
Good example
Probe the approved teaching analyzer on the representative file through verified read-only syntax/definition behavior. Check actual function/location/diagnostics. Unsupported behavior remains a capability gap. Do not infer execution from installation/help/fork origin or introduce unapproved mutations.
Why the change matters
A smoke input directly tests the claim while presence/config remain provisional. Matching results prevent empty success from masquerading as parser support.
Observable expectation
The illustrative probe finds the real function or a supported diagnostic, not full-project coverage. A harmless unique-marker probe can separately check context loading.
Each establishes its own bounded condition only; no frozen probe executes here.
Limits
Representative scope and side effects matter; avoid production actions as casual tests. Tool self-description assists but registered/observed behavior supports claims. Frozen inheritance observations are not universal guarantees.
Sources and evidence
- obra/superpowers · Unique marker test
File at this version8ca22dba9a94 - obra/superpowers · Hook system versus specific event
File at this version8ca22dba9a94