quality
Quality is reported one dimension at a time. There is no composite score: a single 0–100 number would hide which defect an arm has, and different uses tolerate different defects.
| Tier | Applies to | Supported when | Contradicted when |
|---|---|---|---|
| T1 deterministic | reference items | the template's independent Monte-Carlo / SymPy / coverage check reproduces the key | check fails (item rejected at build) |
| T2 program-of-thought | items with four numeric options | a PoT execution matches the key and no other option | executions match only other options, or the key and another option |
| T3 answering proxy | all other items | the reasoning-enabled answer and the stem-only free response both select the key | both select a non-key option |
correctness
structural flags
Nineteen deterministic flags in five groups. Cue flags mark features a test-wise examinee can exploit; style flags mark item-writing guideline violations that are not validity defects on their own.