State the headline accuracy against an indicator

What the unit asks

The draft defines the headline as the share of correctly classified people observed in that run. It uses balanced accuracy, the mean of the true positive rate and the true negative rate, and notes that this equals ordinary accuracy only when the test set is balanced. The statement has to name what was tested, because a component result and a system result are different claims.

What good practice looks like

Report the figure, the two rates, the class balance, the sample size and the indicator. Say whether the object was a component or a system. Do not publish a percentage without the indicator beside it.

Evidence an assessor asks for

What a good answer sounds like

A good answer to P3-04 names The headline statement, with the indicator label and the object description., dates it, and says what it does not cover. A policy or a promise is not an answer. If P3-04 has no artefact, the row is not met and the absence is the finding.

Where comparisons commonly fail

• The headline is an ordinary accuracy on an unbalanced set, presented as balanced accuracy.
• The figure is reported without the indicator of confidence.
• The object is described as the service when only one component was tested.

Questions to put to the supplier

1. What are the true positive and true negative rates behind the average?
2. Which indicator of confidence does this figure claim, and who set the threshold?

How this is used in the kit

P3-04 becomes one row of the comparison register, with an owner, a status and the evidence named above. A status of met asks for the artefact and its date, and the free check uses the same row as an evidence question, so the page and the register share one vocabulary.