Report binary measures at the decision threshold

What the unit asks

The conversion is a policy choice, and it changes the result. A system that passes anyone estimated within two years of the threshold behaves differently from one that passes only a confident estimate. The draft asks for the binary analysis to sit beside the continuous one, so a reader can see both the estimate and the decision.

What good practice looks like

Publish the confusion matrix with counts, not only percentages. State the threshold and the tie rule. Report the two error directions separately, because the harm from a false accept and a false reject is rarely the same.

Evidence an assessor asks for

What a good answer sounds like

A good answer to P3-07 names The confusion matrix with counts and the stated threshold., dates it, and says what it does not cover. A policy or a promise is not an answer. If P3-07 has no artefact, the row is not met and the absence is the finding.

Where comparisons commonly fail

• The report gives one accuracy figure and no matrix.
• The false reject rate is omitted because the buyer only asked about safety.
• The threshold used in the test differs from the live threshold.

Questions to put to the supplier

1. How many false accepts and false rejects, in counts?
2. Who set the conversion rule, and what does each error cost?

How this is used in the kit

P3-07 becomes one row of the comparison register, with an owner, a status and the evidence named above. A status of met asks for the artefact and its date, and the free check uses the same row as an evidence question, so the page and the register share one vocabulary.