Why teachers need explainable AI, not just accurate AI — building the KC dashboard

NumPath redesigned its teacher dashboard to replace a single "7-day accuracy" score with a Knowledge Component (KC) mastery panel that shows color-coded progress bars for individual skills (e.g., borrowing, place value). The authors chose to display a three-tier label system (Novice/Developing/Mastered) with the option to expand for raw data, arguing that this provides actionable information—such as identifying that a student needs targeted borrowing practice—rather than just a vague struggling indicator. The dashboard also includes access controls allowing teachers to view any student's data while students can only see their own, and the backend uses a left-join strategy for cleaner, testable code.

What We Built NumPath's teacher dashboard previously showed one number per student: 7-day accuracy. A teacher looking at "Emma — 43%" has no idea whether Emma is struggling with borrowing, place value, number sense, or all three. The number is technically correct and completely unactionable. In this post I'll walk through how we added a Knowledge Component KC mastery panel to the dashboard — colour-coded progress bars per skill that expand to show p mastery %, mastery level label, and opportunity count. The backend piece is a single endpoint backed by a left-join use case. The research reason it matters is more interesting than the code. The Design Decision The core choice was: what data does a teacher actually need? We had three options: - Accuracy-only what we had : fast to compute, no additional queries, but unactionable - Raw BKT parameters : show p mastery , p learn , p guess , p slip — complete but overwhelming for a classroom teacher - KC mastery levels : translate p mastery into a three-tier label Novice / Developing / Mastered with colour coding, keeping the raw number available on expand We chose option 3. The mastery level thresholds are defined as named constants in get kc states.py : python MASTERY DEVELOPING = 0.40 MASTERY MASTERED = 0.80 def mastery level p mastery: float - str: if p mastery = MASTERY MASTERED: return "Mastered" if p mastery = MASTERY DEVELOPING: return "Developing" return "Novice" One deliberate UX choice: a student with no attempts at all still sees all 5 skills at 0% / Novice. There's no "no data yet" placeholder. The teacher sees the full KC grid from day one — an empty bar is information "this student hasn't encountered this skill yet" , not an error. The access control pattern is worth noting too. We added a require authenticated dependency — any valid JWT — and enforced role logic in the route handler: @router.get "/{student id}/kc-states", response model=KCStatesResponse async def get kc states student id: uuid.UUID, db: AsyncSession = Depends get db , auth: dict = Depends require authenticated , - KCStatesResponse: role = auth.get "role" if role == "student" and auth.get "sub" = str student id : raise HTTPException status code=status.HTTP 403 FORBIDDEN, detail="Access denied" ... Students see their own KC states. Teachers see any student's. The rule lives in one place — the route — rather than being split across two separate dependency functions. Why It Matters for the Research The MacLellan ITS framework's "Teacher-in-the-Loop" principle isn't just about giving teachers a screen. It's about giving them information they can act on . A 43% accuracy number tells a teacher "this student is struggling." A KC panel that shows SUB BORROW at 12% Novice, 8 attempts while PLACE VALUE is at 67% Developing, 14 attempts tells a teacher "this student needs targeted borrowing practice — and they've already tried eight times, so hints aren't landing." That's the difference between a reporting tool and a teaching tool. The RCT we're designing in Phase 4 will measure whether teachers who have KC-level visibility actually intervene differently than those who see accuracy alone. This dashboard is the instrument we're studying, not just a convenience feature. What We Learned The left-join strategy — two separate queries plus a dict lookup — turned out to be cleaner than an ORM outerjoin . SQLAlchemy async outerjoin with nullable columns requires explicit handling of None values in ways that are easy to get wrong. Two queries and a dict.get with a default is more readable and easier to test with mocks: kc by skill id = {record.skill id: record for record in kc records} summaries = KCStateSummary skill code=skill.code, p mastery=round kc by skill id skill.id .p mastery, 3 if skill.id in kc by skill id else 0.0, ... for skill in all skills Nine unit tests covering the use case ran in 0.03s with no live database. That's the payoff for keeping the domain logic in a use case rather than inline in the route. What's Next Phase 2 of the KC dashboard adds recent attempt history to the student detail panel — the specific problems a student got wrong, with their classified mistake codes, so a teacher can see patterns as they form. Key Takeaways - Accuracy is output, KC mastery is signal — a single accuracy number is not enough for a teacher to act; per-KC mastery state is the minimum viable explainability for an ITS - Empty is informative, not broken — showing all KCs at 0% for a new student tells a teacher "this skill hasn't been practised yet"; hiding it implies the data is missing - Two queries + dict one complex join — for small, static reference data 5 skills , two simple queries and a dict lookup are more readable, testable, and maintainable than an ORM outer join with nullable column handling