Skip to content

Exam Question Bank (PYQ corpus) — data inventory

Discovered 2026-06-23 while scoping the Personalized Learner Journey Slice 2 (Study Plan). This is the PYQ "trust anchor" for a weightage-driven study plan. Captured so we don't re-discover it.

TL;DR — where the questions actually live

The 40-years NEET/JEE previous-year-question corpus is NOT in the app DB (vidyanet) and NOT yet in Qdrant. It is a fully-extracted, topic-tagged question bank in a separate database on the same Cloud SQL instance:

  • Instance: vidyanet-postgres-prod (vidyanet-prod:asia-south1:vidyanet-postgres-prod)
  • Database: knowledge_base_status (23 MB) — separate from the app's vidyanet DB (18 MB, near-empty: ~20 users)
  • Connect with the kwilo-prod-query recipe but psql ... -d knowledge_base_status.
  • Qdrant kwilo_exam_questions_v1 is empty (0 points) — ingestion from this DB has not run. Qdrant ncert_multimodal_v1 has NCERT textbooks (12,363 chunks, tagged subject/chapter/topic_label/class 1-12) — good for content + a topic taxonomy, but textbooks, not questions.

Tables (row counts as of 2026-06-23)

Table Rows Notes
questions 6,493 The bank. Fully tagged (see schema).
answers 6,493 1:1 with questions.
explanations 6,493 1:1 with questions.
topics 5,033 Hierarchical: parent_topic_id + level, under chapter_id.
question_assets 1,107 Figures/images for questions (has_figure).
chapters 109 Per book/subject/exam, with chapter_no + PDF page ranges.
exam_books 5 Source books (below).
chunk_extraction 827 Pipeline status (PDF → chunks).
vector_store_ingestion 817 Pipeline status (→ vector store).
exam_chapter_extraction 110 Pipeline status.
jee_questions / neet_questions / ias_questions 0 Empty — superseded by the unified questions table. (ias_questions hints at planned IAS support.)
graph_ingestion / bm25_ingestion 0 Not run.

questions schema (key fields)

id, external_id, book_id, chapter_id, topic_id, subtopic_id, subject, exam, exam_year, exam_session, question_text, options(jsonb), question_type, complexity, page_ref, has_figure, source_qno

  • 100% topic-tagged (topic_id null count = 0 across all rows).
  • exam_year ~99% populated (a few dozen nulls), spanning 1978–2020.
  • complexity populated: medium 2,866 / easy 2,443 / hard 1,179 (5 blank).
  • answers, explanations, topics, chapters join by id / *_id.

Source books (exam_books)

Exam Subject Title Source blob
NEET Chemistry 30 Years NEET/AIPMT Chapterwise Solutions Chemistry NEET_chapterwise_questions_and_solutions/…
NEET Chemistry 33 Years NEET/AIPMT Chapterwise Chemistry (MTG, 2020) same prefix
JEE Chemistry 43 Years Chapterwise Topicwise IIT-JEE Chemistry (1979–2021) JEE_CHapterwise_questions_and_solutions/…
JEE Mathematics 43 Years Chapterwise Topicwise IIT-JEE Mathematics same prefix
JEE Physics 43 Years Chapterwise Topicwise IIT-JEE Physics same prefix

Source PDFs are in a blob store (source_blob_path); figures referenced from an Azure container (kwilojeeharvest…blob.core.windows.net).

Coverage matrix (exam × subject) and the GAP

exam (raw) subject questions year span
JEE Chemistry 2,092 1978–2020
AIPMT Chemistry 1,408 1980–2015
JEE Physics 1,228 1978–2020
JEE Mathematics 1,227 1978–2020
NEET Chemistry 505 1988–2020
IIT JEE Chemistry 30 1979–2005
Mains Chemistry 3 2011–2012

Data-quality caveat — the exam field is not normalized. It must be folded into an exam family before computing weightage: - JEE family = JEE + IIT JEE + Mains (Chemistry 2,125 · Physics 1,228 · Mathematics 1,227) - NEET family = NEET + AIPMT (Chemistry 1,913) (AIPMT was NEET's pre-2017 name.)

Coverage gap (important for the study plan): - JEE: complete — Physics, Chemistry, Mathematics all present. - NEET: Chemistry only. No NEET Biology (which is 360/720 marks — the single biggest NEET section) and no NEET Physics. A NEET study plan today can only weight Chemistry.

Weightage recipe (the trust anchor — proven, $0)

Chapter weightage = count of tagged PYQs per chapter, filtered by exam family + subject. Verified working:

-- Top chapters by historical question count (JEE Physics example)
SELECT c.name AS chapter,
       COUNT(q.id) AS questions,
       ROUND(100.0*COUNT(q.id)/SUM(COUNT(q.id)) OVER (),1) AS pct
FROM questions q
JOIN chapters c ON c.id = q.chapter_id
WHERE q.exam IN ('JEE','IIT JEE','Mains')   -- normalize to JEE family
  AND q.subject = 'Physics'
GROUP BY c.name
ORDER BY questions DESC;

Demo result (JEE Physics, 2026-06-23): Optics 12.9% · Heat & Thermodynamics 11.6% · Electrostatics 10.4% · Magnetics 7.7% · Wave Motion 7.4% · Current Electricity 6.9% · Centre of Mass 6.3% · Rotation 6.2% · … — i.e. ~6 chapters ≈ 50% of the section. This is exactly the topper/coaching weightage table, computed from Kwilo's own corpus.

For recency-weighting and topic-level (not just chapter) weightage, group by topic_id and/or filter exam_year >= <cutoff>.

What this means for Slice 2 (Study Plan)

  • The feared "Slice 2a data foundation" (extract → taxonomy → tag) is largely already done — questions are extracted, chapter/topic/subtopic-tagged, year-stamped, complexity-rated, with answers + explanations.
  • Remaining foundation work is exposure + normalization, not extraction:
  • Make knowledge_base_status queryable from the app (cross-DB read service, or a periodic sync into the vidyanet app DB / the empty kwilo_exam_questions_v1 Qdrant collection).
  • Normalize exam → exam_family (JEE/NEET) at the query/ingest layer.
  • (Optional) compute + cache per-chapter/topic weightage (+ recency weighting).
  • Then the research-backed plan (weightage map → weightage×weakness queue → diagnose/master/simulate → daily card + spaced revision) rides on top, delivered through the existing per-intent wizards.
  • Honest gap to flag in the plan: NEET coverage is Chemistry-only; a NEET plan can't yet weight Biology/Physics. JEE is fully covered.

Reproduce

Run scripts/exam-bank-stats.sh (uses the kwilo-prod-query connection recipe, read-only).