Skip to content

Practice tests: retakeable Artifacts

Every Practice Hub session a learner starts is saved as a practice-test Artifact they can reopen, review question-by-question, and retake. The core contract: a retake is a new attempt on the same frozen questions of the same artifact — it never creates a new artifact, and the review always shows the learner's latest run with a percentage delta versus the previous one. Reuses the existing practice-session tables; no new tables. Related: Practice Player (the run UI it reuses), Practice Hub Engine (how a session's questions are chosen), and the Artifacts federation it plugs into.

Why this exists

  1. Who asked — self-initiated by the product owner, grounded in observed Practice Hub usage.
  2. User pain — a learner does 10 Optics questions, sees "30% accuracy" once, and it evaporates. There's no way to revisit which questions they got wrong, and no way to see whether they're improving. The attempt data was already stored, just invisible.
  3. Cost of not doing it — practice stays ephemeral; no visible improvement loop; the mistake data we already persist never helps the learner. Weaker motivation and retention.
  4. Validated or a guess — partial. The accuracy ring shows learners value the signal; "they'll revisit and retake" is a reasonable hypothesis, not yet validated. Watch re-open and retake rates after launch.
  5. How we'll know it worked — learners reopen saved tests and retake them; attempt-2 % beats attempt-1 % for a meaningful share. Sean Ellis test: if we removed "my practice history / retake" a week after shipping, would learners notice and ask for it back?

How it works

A practice_session row is the test/artifact — it freezes its question set at creation. A retake is a new run of that same set, tracked by attempt_number on each answer plus an explicit current_attempt pointer on the session.

Start (Practice Hub)         Backend                          Review (Artifacts)
  │── POST /practice-sessions ─▶│  create session
  │   {subject, chapter_id}      │  current_attempt = 1
  │                             │  test_number = N (per user+subject+chapter)
  │◀──── questions (run 1) ──────│
  ├── answer each Q ───────────▶│  POST /{id}/attempts
  │                             │  writes attempt_number = current_attempt (1)
  │                             │  completed_at set when run 1 fully answered
  │
  │              …later, from the Artifacts card…
  │                             │◀── GET /practice-sessions/{id} ──│ open review
  │                             │  per-Q results scoped to current run
  │                             │  + attempt history (all runs) + title
  │                             │─── title, testNumber, attempts[], Q results ─▶│
  │                             │
  ├── Retake ──────────────────▶│  POST /{id}/retake
  │                             │  current_attempt += 1 (→ 2), completed_at = null
  │◀──── same questions (blank) ─│  (run 2 has no answers yet → reads blank)
  ├── answer again ────────────▶│  writes attempt_number = 2
  │                             │◀── GET again ──│ review shows run-2 ticks + Δ%

Key behaviors:

  • Retake never mints a new artifact. It bumps current_attempt and re-runs the same frozen question_ids. The card, review, and history all stay on the one artifact.
  • Explicit run pointer, not MAX(attempt_number). current_attempt is a column on the session. Submit writes at it; the summary GET scopes per-question answers to it. This is deliberate: inferring the run from MAX(attempt_number) would make the first retake answer collide with the completed run's unique key, and would leak the prior run's answers into a fresh retake.
  • The summary GET serves the current run. A freshly-retaken run reads blank (so the player starts clean and a refresh resumes correctly); a completed run shows its ✓/✗. The card and attempt history instead use the last run with activity, so a card never shows a blank score just because a retake was opened and abandoned.
  • Only sessions with ≥1 attempt are artifacts. A started-but-unanswered session never clutters the Artifacts list.
  • Ownership everywhere. Submit, retake, summary, and delete all go through the owner-checked session load; the free-tier chapter gate is enforced on create, not re-charged on retake (a retake re-runs an already-owned test).
  • Concurrency-safe counters. Retake and submit load the session FOR UPDATE, so concurrent retakes/submits on the same session serialize on the row lock (no lost current_attempt increment, no answer landing in a stale run). Create takes a transaction-scoped advisory lock on (user, subject, chapter) before counting, so two simultaneous starts can't stamp the same test_number.

Data model

No new tables. Three columns on practice_sessions, one on practice_attempts.

practice_sessions (= the test / artifact)      practice_attempts (= one answer)
  id                                             id
  user_id, subject, chapter_id, chapter_name     session_id, user_id, question_id, chapter_id
  question_ids  (frozen set)                      selected_key, correct_key, is_correct
  title           NEW  (nullable override)        attempt_number  NEW  (which run: 1, 2, …)
  test_number     NEW  (per user+subject+chapter) UNIQUE(session_id, question_id, attempt_number)
  current_attempt NEW  (the run in progress)
  completed_at
  • current_attempt — 1 on create, +1 on retake. The run the learner is currently on.
  • attempt_number — stamped on each answer = the session's current_attempt at submit time. The unique key now includes it, so the same question can be answered once per run.
  • title / test_number — see Naming.

Per-run score is derived (GROUP BY attempt_number), not stored. Mastery/coverage in the Practice Hub Engine are unaffected structurally: coverage stays DISTINCT question_id; mastery counts every attempt (see Non-goals).

Naming

Because one subject+chapter can hold many tests, each carries a short handle plus disambiguating metadata:

  • Default handle: {chapter} — Test {test_number} (e.g. "Optics — Test 2"), where test_number is a per-(user, subject, chapter) counter stamped at create.
  • title is a nullable override (for a future rename UI, or an AI-generated test's own name).
  • The distinguishing detail (subject, source, date) lives in the card's metadata chips, not crammed into the handle — so the handle stays short and multi-topic later just reads "Optics +2".

Artifacts surface

Practice tests federate into the existing Artifacts read view under the practice_test type, tagged with a source: "practice_session" discriminator (the AI-Tutor chat mock tests are the same type, source: "mock_test"). The card shows title · subject · latest correct/total + %; clicking a practice_session card opens the review screen at /practice/test/:id. Delete is owner-scoped and cascades to attempts.

Non-goals and rejected alternatives

  • A new practice_tests table (rejected). An earlier design added a Test→Run→Answer three-table hierarchy. Rejected as over-built: practice_sessions already freezes a question set, so one column (attempt_number) plus an explicit current_attempt expresses retakes with far less migration surface.
  • Storing hub tests in mock_tests (rejected). mock_tests embeds AI-generated questions as JSONB and is single-attempt and conversation-bound — a poor fit for corpus-referenced, retakeable PYQ tests. It stays as-is for chat mock tests.
  • Inferring the run from MAX(attempt_number) (rejected). See "explicit run pointer" above — it collides and leaks answers.
  • Multi-topic and AI-generated tests (out of scope). The schema is left open (chapter_ids as a conceptual array, a source axis), but the UI ships single-chapter PYQ only.
  • Mastery double-count (accepted, not a bug). Retaking a question counts every attempt toward chapter mastery — deliberate: more practice is more signal, and getting a question right on retake correctly lifts mastery. Coverage stays distinct, so retakes never inflate it.

Known gaps (deferred)

  • Mid-abandoned-retake review. If a learner opens a retake and leaves before finishing, the review shows questions neutrally as "not answered" (the current run is empty) while the header shows the last completed run's score. A fuller fix — the review showing the last-answered run per question via a ?view=review parameter — is deferred.
  • Card "attempts" pill. The card doesn't yet show an attempt count; needs an attempt_count field on the artifact item.
  • Rename UI. title override exists in the schema; no UI to set it yet (auto-names only).

Changelog

  • 2026-07-10 — Initial version. Retakeable practice tests saved as Artifacts (review screen, retake, attempt history) shipped in MySetu-AI/kwiloai_webapp #1313. Retroactive-but-concurrent: authored alongside the implementation PR, not after.