feat: surface juror-balanced scores and AI calibration advisory
All checks were successful
Build and Push Docker Image / build (push) Successful in 7m27s

Adds a shared juror-balancing utility (z-score normalization per juror,
rescaled back onto the raw 1-10 scale) and wires it into:

- Admin reports page: Top-10 project table now shows "Raw Avg" and
  "Balanced" columns side by side, and the summary stats row shows a
  balanced-average tile. Sort defaults to balanced so harsh and lenient
  graders no longer skew the ranking.
- Ranking dashboard: each project row shows a green/amber balanced-score
  chip next to the raw average when the two differ by ≥0.05, making it
  obvious when juror calibration moved a project's effective ranking.

Also adds AI Juror Calibration Advisory — a mutation that takes
anonymized per-juror stats, calls OpenAI, and produces a plain-language
explanation of the cohort's grading patterns plus per-juror severity
(normal / notable / outlier) with a one-sentence narrative. The advisory
describes the statistical balance that already runs; it does not
introduce a new weighting layer. Rendered as a panel in the Juror
Consistency tab when a specific round is selected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt
2026-04-24 16:19:00 +02:00
parent 07dd7a0692
commit 982d5193c5
7 changed files with 774 additions and 65 deletions

View File

@@ -82,6 +82,7 @@ type SortableProjectRowProps = {
entry: (RankedProjectEntry & { originalIndex?: number }) | undefined
projectInfo: ProjectInfo | undefined
jurorScores: JurorScore[] | undefined
balancedScore: number | null
onSelect: () => void
isSelected: boolean
originalRank: number | undefined // from snapshotOrder — always in sync with localOrder
@@ -95,6 +96,7 @@ function SortableProjectRow({
entry,
projectInfo,
jurorScores,
balancedScore,
onSelect,
isSelected,
originalRank,
@@ -199,11 +201,25 @@ function SortableProjectRow({
</span>
) : null}
{/* Average score */}
{/* Raw + balanced averages shown side by side */}
{entry?.avgGlobalScore !== null && entry?.avgGlobalScore !== undefined && jurorScores && jurorScores.length > 1 && (
<span className="text-xs font-medium text-muted-foreground" title="Average score">
= {entry.avgGlobalScore.toFixed(1)}
</span>
<div className="flex items-center gap-1.5 text-xs" title="Raw juror average vs. juror-balanced average (z-score normalized per juror, rescaled to 1-10)">
<span className="font-medium text-muted-foreground">
{entry.avgGlobalScore.toFixed(1)}
</span>
{balancedScore != null && Math.abs(balancedScore - entry.avgGlobalScore) >= 0.05 && (
<span
className={cn(
'font-semibold tabular-nums rounded px-1.5 py-0.5 border',
balancedScore > entry.avgGlobalScore
? 'bg-emerald-50 text-emerald-700 border-emerald-200'
: 'bg-amber-50 text-amber-700 border-amber-200',
)}
>
{balancedScore.toFixed(1)}
</span>
)}
</div>
)}
{/* Advance decision indicator */}
@@ -909,7 +925,8 @@ export function RankingDashboard({ competitionId: _competitionId, roundId }: Ran
currentRank={index + 1}
entry={rankingMap.get(projectId)}
projectInfo={projectInfoMap.get(projectId)}
jurorScores={evalScores?.[projectId]}
jurorScores={evalScores?.byProject[projectId]}
balancedScore={evalScores?.balanced[projectId]?.balancedAverage ?? null}
onSelect={() => setSelectedProjectId(projectId)}
isSelected={selectedProjectId === projectId}
originalRank={hasReorders ? snapshotOrder[projectId] : undefined}