Set the sample, the sampling frame and the demographics

What the unit asks

The draft puts sample size and demographics in the planning stage, before the test. The reason is statistical: a claim at the enhanced or strict level is a claim about rare failures, and rare failures need large numbers. A convenient sample of a few hundred people cannot support a ninety-nine point nine per cent claim, however clean the result.

What good practice looks like

Calculate the sample size from the failure rate the indicator implies. Recruit to the frame, not to convenience. Report the achieved sample by band and the groups that fell short. Say what the sample cannot support.

Evidence an assessor asks for

What a good answer sounds like

A good answer to P3-10 names The sample size calculation, with its assumptions., dates it, and says what it does not cover. A policy or a promise is not an answer. If P3-10 has no artefact, the row is not met and the absence is the finding.

Where comparisons commonly fail

• The sample is sized by budget rather than by the failure rate claimed.
• Children, older people or one demographic group are missing from the crew.
• The test is run on one device and one lighting condition, and the report does not say so.

Questions to put to the supplier

1. How many people would this claim need, and how many were tested?
2. Who is in the sample, and who is missing?

How this is used in the kit

P3-10 becomes one row of the comparison register, with an owner, a status and the evidence named above. A status of met asks for the artefact and its date, and the free check uses the same row as an evidence question, so the page and the register share one vocabulary.