Select the test methods and the test crew

What the unit asks

The draft lists the benchmarking steps: name the approach, identify the relevant error rates, plan the execution, document the plan, acquire the crew, run the test and evaluate the results. The independence of the crew is not a courtesy. A crew drawn from the developer's staff or its early users tests a different population from the one the service will meet.

What good practice looks like

Write the plan before recruiting. Record the recruitment route, the exclusion rules and the incentive. Keep the raw data and the analysis code so another analyst can repeat the run.

Evidence an assessor asks for

What a good answer sounds like

A good answer to P3-09 names The signed test plan, dated before the first test., dates it, and says what it does not cover. A policy or a promise is not an answer. If P3-09 has no artefact, the row is not met and the absence is the finding.

Where comparisons commonly fail

• The plan is written after the test as a description of what happened.
• The crew is recruited through the developer's own channels.
• The analysis is run by the supplier on data it selected.

Questions to put to the supplier

1. Who chose the crew, and what did they know about the system?
2. Could another analyst reproduce this result from what is published?

How this is used in the kit

P3-09 becomes one row of the comparison register, with an owner, a status and the evidence named above. A status of met asks for the artefact and its date, and the free check uses the same row as an evidence question, so the page and the register share one vocabulary.