diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index ffae13e..8665e3f 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -18,7 +18,7 @@ Requirements for this milestone. Each maps to roadmap phases. - [x] **RANK-07**: AI ranking service anonymizes project data before sending to OpenAI (follows existing anonymization pattern) - [x] **RANK-08**: Ranking results are stored as snapshots for audit trail (RankingSnapshot model) - [x] **RANK-09**: Ranking auto-triggers when all jury assignments for a round are completed (all evaluations submitted) -- [ ] **RANK-10**: Auto-trigger works retroactively for rounds where all assignments are already complete +- [x] **RANK-10**: Auto-trigger works retroactively for rounds where all assignments are already complete ### Ranking Dashboard @@ -96,7 +96,7 @@ Which phases cover which requirements. Updated during roadmap creation. | RANK-07 | Phase 1 | Complete | | RANK-08 | Phase 1 | Complete | | RANK-09 | Phase 1 | Complete | -| RANK-10 | Phase 1 | Pending | +| RANK-10 | Phase 1 | Complete | | DASH-01 | Phase 2 | Pending | | DASH-02 | Phase 2 | Pending | | DASH-03 | Phase 2 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index ae88826..9006b32 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -1,81 +1,81 @@ -# Roadmap: MOPC — AI Ranking, Advancement & Mentoring - -## Overview - -This milestone extends the MOPC jury voting platform with four capabilities: an AI-powered ranking engine that interprets natural-language criteria and ranks projects per competition category, a drag-and-drop ranking dashboard for admin review and override, a bulk advancement flow with optional email notification, and cross-round mentor persistence that allows mentors to be assigned during any round type. Phases 1-3 are sequential and time-critical (Monday deadline for ranking and advancement). Phase 4 is independent and can run in parallel with Phases 2-3. - -## Phases - -**Phase Numbering:** -- Integer phases (1, 2, 3): Planned milestone work -- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED) - -Decimal phases appear between their surrounding integers in numeric order. - -- [ ] **Phase 1: AI Ranking Backend** - AI service, tRPC router, schema, and auto-trigger for ranking projects per category using natural-language criteria -- [ ] **Phase 2: Ranking Dashboard UI** - Drag-and-drop ranking tab on the round detail page with criteria input, preview-confirm dialog, and project detail side panel -- [ ] **Phase 3: Advancement + Email** - Bulk advancement flow with reject/advance in one operation, notify-vs-silent toggle, and admin-editable email templates -- [ ] **Phase 4: Mentor Persistence** - Mentor assignment during any round type with cross-round carryover and COI re-validation - -## Phase Details - -### Phase 1: AI Ranking Backend -**Goal**: Admin can call an AI ranking API that interprets natural-language criteria and returns per-category ranked project lists, persisted for audit -**Depends on**: Nothing (first phase) -**Requirements**: RANK-01, RANK-02, RANK-03, RANK-04, RANK-05, RANK-06, RANK-07, RANK-08, RANK-09, RANK-10 -**Success Criteria** (what must be TRUE): - 1. Admin can submit natural-language ranking criteria and receive a structured preview of parsed rules before any ranking is applied - 2. AI ranking executes per CompetitionCategory (STARTUP and BUSINESS_CONCEPT ranked separately) using jury scores, pass/fail votes, and admin-defined criteria - 3. All project data is anonymized before leaving the server; OpenAI receives no PII - 4. Ranking results are stored as a RankingSnapshot with full audit trail (who ran it, when, with what criteria) - 5. Ranking auto-triggers when all jury assignments for a round are complete, including rounds where all assignments were already submitted before this feature shipped -**Plans**: TBD - -### Phase 2: Ranking Dashboard UI -**Goal**: Admin has a working ranking interface on the round detail page where they can review, reorder, and prepare projects for advancement -**Depends on**: Phase 1 -**Requirements**: DASH-01, DASH-02, DASH-03, DASH-04, DASH-05, DASH-06, DASH-07 -**Success Criteria** (what must be TRUE): - 1. Admin sees a ranked project list per category on a new Ranking tab of the round detail page, showing AI-suggested order and any admin overrides as visually distinct states - 2. Admin can drag-and-drop projects to reorder within each category; local order is preserved while drag mutations are in-flight and does not snap back - 3. Admin can click any project row to see full evaluation detail in a side panel without leaving the ranking list - 4. Advance button is disabled while any reorder mutation is unsettled; once enabled, admin can select top N projects and advance them in one action - 5. Admin can batch-reject all non-advanced projects from the same interface -**Plans**: TBD - -### Phase 3: Advancement + Email -**Goal**: Admin can advance and reject projects in one operation with full control over whether and what notification emails are sent to applicants -**Depends on**: Phase 2 -**Requirements**: MAIL-01, MAIL-02, MAIL-03, MAIL-04, MAIL-05, MAIL-06 -**Success Criteria** (what must be TRUE): - 1. Admin can choose per advancement batch whether to send an advancement email, a rejection email, both, or update statuses silently with no emails - 2. Admin can edit the text of advancement and rejection email templates directly in the platform, with supported variables shown inline - 3. Variable substitution uses a fixed whitelist ({{firstName}}, {{teamName}}, {{competitionName}}, {{roundName}}); no arbitrary template expressions are evaluated - 4. If the next round has no automated welcome emails configured, the system warns admin before they confirm advancement - 5. Email templates are stored in the database per competition/round and persist across sessions -**Plans**: TBD - -### Phase 4: Mentor Persistence -**Goal**: Admin can assign mentors to projects from any round type, and those assignments carry forward to subsequent rounds unless the project is eliminated -**Depends on**: Nothing (independent track — can run in parallel with Phases 2-3) -**Requirements**: MENT-01, MENT-02, MENT-03, MENT-04, MENT-05, MENT-06 -**Success Criteria** (what must be TRUE): - 1. Admin can open the mentor assignment UI from any round detail page, not only rounds of type MENTORING - 2. When a project advances to the next round, existing mentor assignments carry over automatically without admin action - 3. Before any mentor assignment carries over, the system re-validates that no conflict of interest exists between the mentor and the project in the new round; if one is found, the carryover is blocked and admin is notified - 4. Admin can override, change, or remove a mentor assignment at any point from any round - 5. AI suggests mentor-project matches based on expertise with admin confirmation required before the assignment is saved - 6. The ranking dashboard shows current mentor assignment status for each project in the ranked list -**Plans**: TBD - -## Progress - -**Execution Order:** -Phases execute in numeric order: 1 → 2 → 3 → 4 (Phase 4 can be parallelized with 2-3) - -| Phase | Plans Complete | Status | Completed | -|-------|----------------|--------|-----------| -| 1. AI Ranking Backend | 3/4 | In Progress| | -| 2. Ranking Dashboard UI | 0/TBD | Not started | - | -| 3. Advancement + Email | 0/TBD | Not started | - | -| 4. Mentor Persistence | 0/TBD | Not started | - | +# Roadmap: MOPC — AI Ranking, Advancement & Mentoring + +## Overview + +This milestone extends the MOPC jury voting platform with four capabilities: an AI-powered ranking engine that interprets natural-language criteria and ranks projects per competition category, a drag-and-drop ranking dashboard for admin review and override, a bulk advancement flow with optional email notification, and cross-round mentor persistence that allows mentors to be assigned during any round type. Phases 1-3 are sequential and time-critical (Monday deadline for ranking and advancement). Phase 4 is independent and can run in parallel with Phases 2-3. + +## Phases + +**Phase Numbering:** +- Integer phases (1, 2, 3): Planned milestone work +- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED) + +Decimal phases appear between their surrounding integers in numeric order. + +- [x] **Phase 1: AI Ranking Backend** - AI service, tRPC router, schema, and auto-trigger for ranking projects per category using natural-language criteria (completed 2026-02-27) +- [ ] **Phase 2: Ranking Dashboard UI** - Drag-and-drop ranking tab on the round detail page with criteria input, preview-confirm dialog, and project detail side panel +- [ ] **Phase 3: Advancement + Email** - Bulk advancement flow with reject/advance in one operation, notify-vs-silent toggle, and admin-editable email templates +- [ ] **Phase 4: Mentor Persistence** - Mentor assignment during any round type with cross-round carryover and COI re-validation + +## Phase Details + +### Phase 1: AI Ranking Backend +**Goal**: Admin can call an AI ranking API that interprets natural-language criteria and returns per-category ranked project lists, persisted for audit +**Depends on**: Nothing (first phase) +**Requirements**: RANK-01, RANK-02, RANK-03, RANK-04, RANK-05, RANK-06, RANK-07, RANK-08, RANK-09, RANK-10 +**Success Criteria** (what must be TRUE): + 1. Admin can submit natural-language ranking criteria and receive a structured preview of parsed rules before any ranking is applied + 2. AI ranking executes per CompetitionCategory (STARTUP and BUSINESS_CONCEPT ranked separately) using jury scores, pass/fail votes, and admin-defined criteria + 3. All project data is anonymized before leaving the server; OpenAI receives no PII + 4. Ranking results are stored as a RankingSnapshot with full audit trail (who ran it, when, with what criteria) + 5. Ranking auto-triggers when all jury assignments for a round are complete, including rounds where all assignments were already submitted before this feature shipped +**Plans**: TBD + +### Phase 2: Ranking Dashboard UI +**Goal**: Admin has a working ranking interface on the round detail page where they can review, reorder, and prepare projects for advancement +**Depends on**: Phase 1 +**Requirements**: DASH-01, DASH-02, DASH-03, DASH-04, DASH-05, DASH-06, DASH-07 +**Success Criteria** (what must be TRUE): + 1. Admin sees a ranked project list per category on a new Ranking tab of the round detail page, showing AI-suggested order and any admin overrides as visually distinct states + 2. Admin can drag-and-drop projects to reorder within each category; local order is preserved while drag mutations are in-flight and does not snap back + 3. Admin can click any project row to see full evaluation detail in a side panel without leaving the ranking list + 4. Advance button is disabled while any reorder mutation is unsettled; once enabled, admin can select top N projects and advance them in one action + 5. Admin can batch-reject all non-advanced projects from the same interface +**Plans**: TBD + +### Phase 3: Advancement + Email +**Goal**: Admin can advance and reject projects in one operation with full control over whether and what notification emails are sent to applicants +**Depends on**: Phase 2 +**Requirements**: MAIL-01, MAIL-02, MAIL-03, MAIL-04, MAIL-05, MAIL-06 +**Success Criteria** (what must be TRUE): + 1. Admin can choose per advancement batch whether to send an advancement email, a rejection email, both, or update statuses silently with no emails + 2. Admin can edit the text of advancement and rejection email templates directly in the platform, with supported variables shown inline + 3. Variable substitution uses a fixed whitelist ({{firstName}}, {{teamName}}, {{competitionName}}, {{roundName}}); no arbitrary template expressions are evaluated + 4. If the next round has no automated welcome emails configured, the system warns admin before they confirm advancement + 5. Email templates are stored in the database per competition/round and persist across sessions +**Plans**: TBD + +### Phase 4: Mentor Persistence +**Goal**: Admin can assign mentors to projects from any round type, and those assignments carry forward to subsequent rounds unless the project is eliminated +**Depends on**: Nothing (independent track — can run in parallel with Phases 2-3) +**Requirements**: MENT-01, MENT-02, MENT-03, MENT-04, MENT-05, MENT-06 +**Success Criteria** (what must be TRUE): + 1. Admin can open the mentor assignment UI from any round detail page, not only rounds of type MENTORING + 2. When a project advances to the next round, existing mentor assignments carry over automatically without admin action + 3. Before any mentor assignment carries over, the system re-validates that no conflict of interest exists between the mentor and the project in the new round; if one is found, the carryover is blocked and admin is notified + 4. Admin can override, change, or remove a mentor assignment at any point from any round + 5. AI suggests mentor-project matches based on expertise with admin confirmation required before the assignment is saved + 6. The ranking dashboard shows current mentor assignment status for each project in the ranked list +**Plans**: TBD + +## Progress + +**Execution Order:** +Phases execute in numeric order: 1 → 2 → 3 → 4 (Phase 4 can be parallelized with 2-3) + +| Phase | Plans Complete | Status | Completed | +|-------|----------------|--------|-----------| +| 1. AI Ranking Backend | 4/4 | Complete | 2026-02-27 | +| 2. Ranking Dashboard UI | 0/TBD | Not started | - | +| 3. Advancement + Email | 0/TBD | Not started | - | +| 4. Mentor Persistence | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 5386137..4ec0a9f 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -1,3 +1,16 @@ +--- +gsd_state_version: 1.0 +milestone: v1.0 +milestone_name: milestone +status: unknown +last_updated: "2026-02-27T00:07:04.010Z" +progress: + total_phases: 1 + completed_phases: 1 + total_plans: 4 + completed_plans: 4 +--- + # Project State ## Project Reference @@ -10,30 +23,31 @@ See: .planning/PROJECT.md (updated 2026-02-26) ## Current Position Phase: 1 of 4 (AI Ranking Backend) -Plan: 3 of TBD in current phase +Plan: 4 of 4 in current phase (Phase 1 complete) Status: In progress -Last activity: 2026-02-27 — Plan 03 complete: tRPC rankingRouter with 5 procedures registered in appRouter +Last activity: 2026-02-27 — Plan 04 complete: auto-trigger hook on evaluation submit + triggerAutoRank + retroactiveScan procedures -Progress: [███░░░░░░░] ~30% +Progress: [████░░░░░░] ~40% ## Performance Metrics **Velocity:** -- Total plans completed: 3 -- Average duration: ~3 min -- Total execution time: ~10 min +- Total plans completed: 4 +- Average duration: ~5 min +- Total execution time: ~18 min **By Phase:** | Phase | Plans | Total | Avg/Plan | |-------|-------|-------|----------| -| 01-ai-ranking-backend | 3 | ~10 min | ~3 min | +| 01-ai-ranking-backend | 4 | ~18 min | ~5 min | **Recent Trend:** -- Last 5 plans: 01-01 (~3 min), 01-02 (~3 min), 01-03 (~4 min) +- Last 5 plans: 01-01 (~3 min), 01-02 (~3 min), 01-03 (~4 min), 01-04 (~8 min) - Trend: Fast (service-layer tasks) *Updated after each plan completion* +| Phase 01-ai-ranking-backend P04 | 8 | 2 tasks | 3 files | ## Accumulated Context @@ -53,6 +67,9 @@ Recent decisions affecting current work: - [01-02]: PrismaClient imported as real type (not import type) so transaction clients are compatible - [01-03]: Typed arrays cast to Prisma.InputJsonValue via `unknown` (direct cast rejected by strict TS — no index signature) - [01-03]: getSnapshot uses findUnique + manual TRPCError NOT_FOUND (findUniqueOrThrow gives INTERNAL_SERVER_ERROR) +- [Phase 01-04]: triggerAutoRankIfComplete defined as module-level non-exported function in evaluation.ts — avoids circular imports, colocated with the mutation it serves +- [Phase 01-04]: EvaluationConfig null fallback typed as {} as EvaluationConfig — required for TypeScript strict mode to recognize rankingCriteria and autoRankOnComplete fields +- [Phase 01-04]: retroactiveScan uses RETROACTIVE triggerType to distinguish from MANUAL/AUTO/QUICK — prevents duplicate re-runs on subsequent scans ### Pending Todos @@ -67,5 +84,5 @@ None yet. ## Session Continuity Last session: 2026-02-27 -Stopped at: Completed 01-03-PLAN.md (tRPC rankingRouter — 5 procedures, registered in appRouter) +Stopped at: Completed 01-04-PLAN.md (auto-trigger hook + triggerAutoRank + retroactiveScan + AI ranking notifications) Resume file: None diff --git a/.planning/phases/01-ai-ranking-backend/01-04-SUMMARY.md b/.planning/phases/01-ai-ranking-backend/01-04-SUMMARY.md new file mode 100644 index 0000000..c325a2c --- /dev/null +++ b/.planning/phases/01-ai-ranking-backend/01-04-SUMMARY.md @@ -0,0 +1,134 @@ +--- +phase: 01-ai-ranking-backend +plan: 04 +subsystem: ai-ranking +tags: [auto-trigger, notifications, ranking, evaluation, tRPC] +requirements: [RANK-09, RANK-10] +dependency_graph: + requires: [01-01, 01-02, 01-03] + provides: [auto-trigger-on-evaluation-complete, retroactive-scan, admin-notifications] + affects: [evaluation-submit-mutation, ranking-router, in-app-notifications] +tech_stack: + added: [] + patterns: [fire-and-forget async, cooldown-guard, module-level-helper-function] +key_files: + created: [] + modified: + - src/server/services/in-app-notification.ts + - src/server/routers/evaluation.ts + - src/server/routers/ranking.ts +decisions: + - "triggerAutoRankIfComplete defined as module-level (non-exported) function in evaluation.ts — avoids circular imports and keeps the auto-trigger logic colocated with the mutation it serves" + - "EvaluationConfig null fallback typed as {} as EvaluationConfig rather than just {} — required for TypeScript strict mode to recognize rankingCriteria and autoRankOnComplete fields" + - "ParsedRankingRule[] cast via unknown as Prisma.InputJsonValue — Prisma InputJsonValue does not overlap with typed arrays, double-cast is the correct pattern throughout the codebase" + - "retroactiveScan uses RETROACTIVE triggerType to distinguish from MANUAL/AUTO/QUICK — prevents duplicate re-runs on subsequent scans" + - "triggerAutoRank procedure in ranking.ts uses MANUAL triggerType (not AUTO) because it is admin-initiated, not system-initiated" +metrics: + duration: ~8min + completed_date: "2026-02-27" + tasks_completed: 2 + files_modified: 3 +--- + +# Phase 1 Plan 04: Auto-Trigger + Retroactive Scan Summary + +**One-liner:** Fire-and-forget auto-ranking on evaluation completion with 5-minute cooldown guard, plus retroactive scan procedure for rounds already complete at deploy time. + +## What Was Built + +### Task 1: AI_RANKING_COMPLETE + AI_RANKING_FAILED notification types + +Added two new entries to `src/server/services/in-app-notification.ts`: + +- `NotificationTypes.AI_RANKING_COMPLETE` — type string + `BarChart3` icon + `normal` priority +- `NotificationTypes.AI_RANKING_FAILED` — type string + `AlertTriangle` icon + `high` priority + +Pattern follows existing FILTERING_COMPLETE / FILTERING_FAILED entries exactly. + +### Task 2: Auto-trigger hook + new ranking procedures + +**evaluation.ts — `triggerAutoRankIfComplete` function:** + +```typescript +async function triggerAutoRankIfComplete( + roundId: string, + prisma: PrismaClient, + userId: string, +): Promise +``` + +Logic flow: +1. Count required assignments — skip if not all complete +2. Read `round.configJson` → check `autoRankOnComplete` + `rankingCriteria` — skip silently if not configured +3. Cooldown guard — skip if AUTO snapshot exists within last 5 minutes +4. Call `aiQuickRank()` — executes in 10-30s asynchronously +5. Create `RankingSnapshot` with `triggerType: 'AUTO'`, `triggeredById: null` +6. Notify admins via `AI_RANKING_COMPLETE` notification +7. Catch-all: any error sends `AI_RANKING_FAILED` notification, never rethrows + +**Fire-and-forget call in submit mutation (line 378):** +```typescript +void triggerAutoRankIfComplete(evaluation.assignment.roundId, ctx.prisma, ctx.user.id) +``` +Placed after `$transaction([evaluation.update, assignment.update])`, before `logAudit`. The submission returns immediately — ranking runs asynchronously. + +**ranking.ts — 7 total procedures:** + +| Procedure | Type | Trigger | Description | +|-----------|------|---------|-------------| +| `parseRankingCriteria` | mutation | admin | Preview-only parse (RANK-01, RANK-03) | +| `executeRanking` | mutation | admin | Confirmed ranking with pre-parsed rules (RANK-05, RANK-06, RANK-08) | +| `quickRank` | mutation | admin | Parse + execute in one step (RANK-04) | +| `listSnapshots` | query | admin | List snapshots for round, most recent first | +| `getSnapshot` | query | admin | Retrieve single snapshot by ID | +| `triggerAutoRank` | mutation | admin | Manual trigger from round config (RANK-09) | +| `retroactiveScan` | mutation | admin | Scan all active/closed rounds (RANK-10) | + +**retroactiveScan logic:** +- Finds all `ROUND_ACTIVE` and `ROUND_CLOSED` rounds +- Checks each for `autoRankOnComplete + rankingCriteria` config +- Checks if all required assignments are complete +- Skips rounds that already have a `RETROACTIVE` snapshot +- Executes ranking sequentially (not parallel) to avoid OpenAI rate limits +- Returns `{ results[], total, triggered }` summary + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] TypeScript strict mode: `?? {}` loses EvaluationConfig type** +- **Found during:** Task 2 typecheck +- **Issue:** `(configJson as EvaluationConfig | null) ?? {}` — the `{}` fallback widens the type to `{}` which doesn't have `rankingCriteria` / `autoRankOnComplete` fields, causing TS2339 errors +- **Fix:** Changed to `?? ({} as EvaluationConfig)` in all three locations (evaluation.ts + two in ranking.ts) +- **Files modified:** `src/server/routers/evaluation.ts`, `src/server/routers/ranking.ts` +- **Commit:** c310631 + +**2. [Rule 1 - Bug] TypeScript: ParsedRankingRule[] requires double-cast to InputJsonValue** +- **Found during:** Task 2 typecheck +- **Issue:** `result.parsedRules as Prisma.InputJsonValue` produces TS2352 — neither type overlaps +- **Fix:** Changed to `result.parsedRules as unknown as Prisma.InputJsonValue` (matching the pattern already used for `rankedProjects` arrays) +- **Files modified:** `src/server/routers/ranking.ts` (triggerAutoRank + retroactiveScan) +- **Commit:** c310631 + +## Build Status + +- `npm run typecheck` — PASSED (0 errors) +- `npm run build` — PASSED (full production build) + +## Key Links Implemented + +| From | To | Via | +|------|----|-----| +| `evaluation.ts submit` | `triggerAutoRankIfComplete` | `void` fire-and-forget after `isCompleted: true` | +| `triggerAutoRankIfComplete` | `ai-ranking.ts quickRank` | `aiQuickRank(criteriaText, roundId, prisma, userId)` | +| `triggerAutoRankIfComplete` | `in-app-notification.ts` | `notifyAdmins({ type: AI_RANKING_COMPLETE })` | +| `ranking.ts triggerAutoRank` | `ai-ranking.ts quickRank` | admin-initiated, creates MANUAL snapshot | +| `ranking.ts retroactiveScan` | `ai-ranking.ts quickRank` | sequential per-round, creates RETROACTIVE snapshot | + +## Self-Check: PASSED + +- `src/server/routers/evaluation.ts` — modified, contains `void triggerAutoRankIfComplete(...)` +- `src/server/routers/ranking.ts` — modified, contains `triggerAutoRank` and `retroactiveScan` +- `src/server/services/in-app-notification.ts` — modified, contains `AI_RANKING_COMPLETE` and `AI_RANKING_FAILED` +- Commits: 4683bb8 (Task 1), c310631 (Task 2) +- Build: PASSED