The most valuable thing the Dartmouth team built wasn't the grader. It was the fact that they could answer "did completing this lesson's quiz correlate with doing better on the exam?" — per module, per format. That question is why they discovered multiple-choice quizzing produced no measurable learning while constructed-response did. Without per-lesson dosage logged against exam outcomes, that finding is invisible, and the platform ships the useless format forever because everyone felt engaged.
This is post 7 of the assessment-first series. It's about the least glamorous and most compounding part of doerkit: the telemetry that lets the platform measure itself.
Most edtech analytics report engagement — logins, time-on-page, questions attempted. Those are vanity metrics; they measure whether people showed up, not whether showing up did anything. The variable the Dartmouth study built its whole argument on is dosage: how many lessons a student actually completed, regressed against exam performance. The distinction is the entire finding: engagement was comparable-or-higher under multiple-choice, but dosage only tracked exam scores under constructed response. If you log engagement you learn nothing; if you log dosage you learn which features work.
So doerkit logs two things from day one: an append-only events
table (lesson views, quiz starts, submissions with score and pass/fail) and an attempts
table (every quiz and review attempt with its score). Both carry a student key and a timestamp. That's the minimal schema, and it's enough to reconstruct dosage-versus-outcome for any cohort you later attach exam scores to.
CREATE TABLE attempts (
id INTEGER PRIMARY KEY, student TEXT, kind TEXT, -- 'lesson' | 'review'
lesson_id TEXT, score REAL, passed INTEGER, created_at TEXT
);
The dosage dashboard rolls this up per student: lessons passed (the dosage number), quiz attempts, average score, reviews passed versus attempted, total events. It's a plain SQL rollup rendered as an HTML table, with no charting library and no analytics vendor. The point isn't the visualization; it's that the raw material for an efficacy analysis exists the moment the first student touches the platform, instead of being a data-collection project you scramble to start after someone asks whether the thing works.
That framing matters for what this platform is for. Efficacy evidence is the currency of institutional edtech sales and the thing every rigorous claim in this space is missing. A platform instrumented for dosage-outcome analysis generates its own evidence base as a byproduct of being used — every cohort makes the next efficacy claim stronger. The data asset compounds; the code doesn't. That's the actual moat in this category, and it costs two database tables to start accruing.
If you're building anything with practice and outcomes, log these from commit one, before you think you need them:
Retrofitting this after launch means the first cohort is unmeasurable, and the first cohort is exactly the one a skeptical instructor asks about. Instrument before you need it, because the need arrives as a question you can't answer retroactively.
Dosage-outcome correlation is not causation, and this is the load-bearing caveat for the whole series: motivated students both complete more lessons and score higher, so raw dosage regressions are selection-inflated. The Dartmouth authors handled it by controlling for prior midterm performance, which brackets the true effect between an over-adjusted 0.71 SD and a selection-inflated 1.30 SD, and doerkit's telemetry can produce the same bracketing only if you feed it exam scores, which it doesn't collect on its own. There's a privacy surface too: a student-keyed event log is FERPA-relevant data the moment this leaves a laptop, so the demo uses a self-chosen name and no real roster, and a genuine deployment needs a data agreement this scope deliberately avoids. Telemetry that measures learning is also telemetry that surveils learners; build it, and own that both are true.
Next post is the capstone: run the whole thing yourself, what an actual institutional deployment would still need, and an honest accounting of where the assessment-first bet holds and where it doesn't.