Reviewer quality
Reviewer standards
Every case is scored by two independent reviewers. When they disagree, a third reviewer adjudicates. Candidates can appeal to a reviewer who had no part in the original decision. These rules are enforced by the platform, not just written down.
Live quality record
Counts come straight from platform records. Zero means nothing has happened yet — no figures are estimated.
0
Approved reviewers
0
Cases double-scored
Not yet measured
Disagreement rate
0 of 0
Disagreements adjudicated
0 of 0
Calibration exercises within tolerance
0 (0) of 0
Appeals decided (upheld)
How consistency is enforced
- Reviewers are approved per skill by an administrator after checking experience and evidence.
- A reviewer receives no live cases until they have passed at least 3 calibration exercises within 10 points of the reference score in the last 6 months, and their most recent exercise was within tolerance.
- Two reviewers score each case without seeing each other's scores.
- A score gap above 15 points, or different recommendations, automatically brings in a third adjudicating reviewer.
- Reviewers must declare conflicts of interest before a case file opens; a declared conflict reassigns the case.
- Appeals are decided by a reviewer approved for the skill who did not review the original case — the platform rejects any other assignment.
Skill-specific reviewer requirements
Published first for the skills under validation. Other skills will follow as their validation reports are written.
SD-01
Frontend Web Development
- Practical experience
- At least 3 years' paid professional experience in Frontend Web Development, including reviewing other developers' work.
- Evidence checked before approval
- CV or LinkedIn profile showing at least 3 years' paid professional work in this skill
- Portfolio or repositories with work the reviewer personally built
- Two professional references or verifiable employer history
- Training
- SkillSignature rubric and scoring training (recorded completion)
- Bias, fairness and reasonable-adjustments briefing
- Confidentiality and data-handling agreement
- Calibration
- Before receiving live cases, the reviewer independently scores 3 consented calibration submissions. Each score must be within 10 points of the reference score agreed by two senior reviewers. Repeated every 6 months and whenever the rubric version changes.
- Conflicts of interest
- Reviewers declare any personal, employment, financial or training relationship with the candidate before the case file opens. A declared conflict removes the reviewer and a replacement is assigned automatically. Reviewers never review their own application.
- Ongoing monitoring
- Every case is scored by two reviewers independently. A score gap above 15 points or different recommendations automatically assigns a third adjudicating reviewer. Reviewers whose calibration falls outside tolerance are paused until retrained.
- Appeals
- Candidates may appeal a decision once per application. Appeals are decided by a reviewer approved for the skill who did not review the original case.
SD-02
Backend Web Development
- Practical experience
- At least 3 years' paid professional experience in Backend Web Development, including reviewing other developers' work.
- Evidence checked before approval
- CV or LinkedIn profile showing at least 3 years' paid professional work in this skill
- Portfolio or repositories with work the reviewer personally built
- Two professional references or verifiable employer history
- Training
- SkillSignature rubric and scoring training (recorded completion)
- Bias, fairness and reasonable-adjustments briefing
- Confidentiality and data-handling agreement
- Calibration
- Before receiving live cases, the reviewer independently scores 3 consented calibration submissions. Each score must be within 10 points of the reference score agreed by two senior reviewers. Repeated every 6 months and whenever the rubric version changes.
- Conflicts of interest
- Reviewers declare any personal, employment, financial or training relationship with the candidate before the case file opens. A declared conflict removes the reviewer and a replacement is assigned automatically. Reviewers never review their own application.
- Ongoing monitoring
- Every case is scored by two reviewers independently. A score gap above 15 points or different recommendations automatically assigns a third adjudicating reviewer. Reviewers whose calibration falls outside tolerance are paused until retrained.
- Appeals
- Candidates may appeal a decision once per application. Appeals are decided by a reviewer approved for the skill who did not review the original case.
SD-03
Full-Stack Web Development
- Practical experience
- At least 3 years' paid professional experience in Full-Stack Web Development, including reviewing other developers' work.
- Evidence checked before approval
- CV or LinkedIn profile showing at least 3 years' paid professional work in this skill
- Portfolio or repositories with work the reviewer personally built
- Two professional references or verifiable employer history
- Training
- SkillSignature rubric and scoring training (recorded completion)
- Bias, fairness and reasonable-adjustments briefing
- Confidentiality and data-handling agreement
- Calibration
- Before receiving live cases, the reviewer independently scores 3 consented calibration submissions. Each score must be within 10 points of the reference score agreed by two senior reviewers. Repeated every 6 months and whenever the rubric version changes.
- Conflicts of interest
- Reviewers declare any personal, employment, financial or training relationship with the candidate before the case file opens. A declared conflict removes the reviewer and a replacement is assigned automatically. Reviewers never review their own application.
- Ongoing monitoring
- Every case is scored by two reviewers independently. A score gap above 15 points or different recommendations automatically assigns a third adjudicating reviewer. Reviewers whose calibration falls outside tolerance are paused until retrained.
- Appeals
- Candidates may appeal a decision once per application. Appeals are decided by a reviewer approved for the skill who did not review the original case.
Want to review? Apply to become a reviewer.