Evaluation Framework

Fast feedback from AI. Final judgement from people.

Every submission is scored against a rubric you can read before submitting. AI produces the report; humans stay accountable for every decision that matters.

The Principles

Five commitments the system is built on.

Transparent criteria

Every rubric — criteria, weights, maximums — is published on the Three-Lock Journey page before you submit.

Criterion-level feedback

Reports break scores down per criterion with strengths, weaknesses and concrete improvement recommendations.

Human verification

Human evaluators verify important decisions. AI never independently decides finalists or winners.

Automatic moderation

When AI and human scores differ significantly, the case routes to moderation review automatically.

A published formula

Season 2 final selection weighed Lock 1 (15%), Lock 2 (20%), Lock 3 (50%) and human review (15%).

The Pipeline

What happens to a submission.

Submit

Your lock submission arrives

Lock 3 submissions first pass automatic link checks — broken or private links return the submission to draft so you can fix them.

Score

AI evaluates against the rubric

The evaluation engine scores each criterion and writes the report: strengths, weaknesses, recommendations.

Review

Humans verify what matters

Independent human evaluators review with blind-first scoring — they score before seeing the AI’s numbers, and conflicts of interest must be declared.

Moderate

Variance triggers moderation

Significant AI-versus-human disagreement flags the case for moderation before anything is finalised.

Decide

Selection and results

The weighted formula plus verified review produced the finalists; the jury confirmed the winners in person at RIT.

Honest Boundaries

What AI does and does not do here.

AI does

Score every submission against the published rubric
Write criterion-level feedback reports
Check submitted links resolve and are accessible
Flag inconsistencies for human attention

AI does not

Select finalists on its own
Decide winners
Override human moderation
See who paid what — evaluation is isolated from payment entirely

Teams disagreeing with a report have structured paths: Lock 1 includes a revision opportunity, and material concerns can be raised through the support desk for human review. Feedback exists to be argued with — the strongest teams engage with it rather than merely accept it.

Evaluation you can trust is evaluation you can read.

The published rubrics remain available as a record of how Season 2 work was evaluated.

View Season 2