# Comparing Accuracy and Interpretability in Knowledge Tracing Models for Mathematical Learning > Source: > Published: 2026-11-05 04:46:43+00:00 Can we figure out how well a student could perform in the future given their current scores? Using knowledge tracing, we can create predictive models that can accurately predict whether or not the student will get the next question in a subject right depending on some different parameters. The two most prominent models are Bayesian Knowledge Tracing and Deep Knowledge Tracing; while BKT uses interpretable parameters to return predicted question correctness and mastery probabilities, DKT uses a black-box recurrent neural network to return accurate predictions about question correctness. In this experiment, we created a BKT and DKT model that was fit on a training set derived from the ASSISTments 2017 student dataset and compared their accuracy and interpretability. While DKT ranked higher in terms of accuracy of predictions, BKT was significantly better when it came to actually understanding the cause-and-effect relationships between the inputs and outputs of the model. When a student answers a practice question, what does that answer tell us about what they actually know? Knowledge tracing tries to answer exactly that. Given a student’s history of answers, a knowledge tracing model estimates their knowledge state and predicts whether they will get the next question right (Abdelrahman et al., 2022). Adaptive learning systems built on these estimates have been linked to better learner mastery (He et al., 2026). Two models are leaders in the field: Bayesian Knowledge Tracing (BKT), which tracks a hidden “learned or not” state for each skill using a handful of interpretable parameters, and Deep Knowledge Tracing (DKT), which passes the answer history through a recurrent neural network (Shen et al., 2024). Both can predict student performance well (Abdelrahman et al., 2022; Shen et al., 2024), but they differ in what they give you beyond a prediction. BKT produces an explicit mastery probability for each skill. DKT’s reasoning is stored in learned weights that are hard to inspect. Prior work suggests a tradeoff in which more flexible models predict better but are harder to understand (Majmudar, 2025). Interpretability matters in practice, because a teacher or system designer has to trust a model’s output to act on it, for example when deciding whether a student has mastered a skill and can move on, whereas accuracy is important to ensure that teachers get information that can be used without errors. In this project, we test that tradeoff directly. We fit a per-skill BKT model and a DKT model on the same ASSISTments 2017 data, hold out unseen students for testing, and ask two questions: which model predicts students’ next answers better, and which is easier to interpret? We measure interpretability with concrete proxies, such as whether BKT’s learned parameters are plausible and how consistently each model’s predictions respond to a student’s answers. The rest of this article covers the data, the methods, the results, and what they mean for choosing between the two models. **Dataset:** For this experiment, we used the ASSISTments 2017 dataset provided for free online by the ASSISTments 2017 Data Mining Competition. The dataset contains rows of students and their performance on quiz questions, along with the subject covered in the question and the time stamps for each question. The original dataset was not cleaned, so a Jupyter Notebook was used to remove unnecessary columns to avoid bloat and sort the dataset by student ID, the start time, and the end time. From this dataset, a training set and test set were created for fitting the models — 80 percent of the students were used for training and 20 percent for testing. **Procedure:** To begin, we used a Jupyter Notebook to read, clean, and split the data into training and test sets. The training and test sets were then exported as .pkl files to a different “data” folder. Afterwards, using PyTorch, python files were created for the BKT and DKT models, defining forward pass and backpropagation functions for each. In a separate folder, fitting files were created to fit the models on the training data, after which the trained model would be saved for each. After creating metric evaluation files for all of our measured metrics (cross-entropy, auc, accuracy), they were saved in one metrics file that returned a dictionary of the metrics for a single function. Finally, everything was run at once in a run_experiment file that began by fitting the models before running them on a test set, measuring their metrics, and creating figures to analyze each metric in comparison between the models. **Materials and Design:** The dataset was the same for both models, and both models were trained and tested on the same training and testing sets, respectively. The only difference maintained was the type of learning model. **Measurements:** The metrics used to compare the performance of both models were cross-entropy loss (using binary cross-entropy), AUC measurements, and prediction accuracy on the test set. Both models were compared on the basis of each of the metrics and were additionally compared in terms of their interpretability. **Predictive Accuracy:** The graphs shown above are all measures of accuracy for both the BKT and DKT models, comparing their performance in different metrics such as per-skill AUC, total AUC, predictive accuracy, RMSE (Root Mean Square Error), and cross-entropy loss measured by binary cross-entropy. These measurements and graphs reflect the accuracy with which the models not only predicted student masteries and correctness for current questions but also predicted future growth in mastery and correctness. **Prediction Stability:** This graph depicts two comparisons: a comparison in direction consistency and one in waviness. Direction consistency measures how well the direction of the model’s prediction update aligns with the student’s improvement while waviness measures how much the model’s mastery estimates fluctuate over time-steps. **Interpretability:** The interpretability graph shows us how BKT and DKT track learning over four skills and across multiple practice opportunities. While DKT often achieves higher predictive accuracy, its correctness estimates fluctuate more sharply from one timestep to the next. In contrast, BKT produces smoother, more stable learning curves and also provides an explicit mastery estimate that increases gradually over time. Results from the experiment demonstrated a concept shown in prior comparisons between BKT and DKT models: the accuracy-interpretability tradeoff (Majmudar, 2025). Though DKT consistently outperformed BKT in all accuracy metrics (cross-entropy loss, per-skill AUC, macro-skill AUC, and predictive accuracy), BKT was significantly better in terms of interpretability metrics (like low waviness and higher directional consistency). Not only are these results consistent with current research, but they also make sense based on the way these models are built. Bayesian Knowledge Tracing, as the name suggests, is patterned on Bayes’s Theorem and uses four transparent parameters (probability of guess, probability of slip, probability of learn, and probability of initial mastery) in the theorem to create a model function that is more clear for the teachers or administrators using it. Meanwhile, DKT uses an LSTM-based recurrent neural network to model temporal dependencies in student learning. However, in the process of improving accuracy, DKT’s model becomes more of a black-box model and is much harder to understand from a human point-of-view. Therefore, we see that DKT is much better in terms of accuracy in predictions of correctness and mastery trends, but BKT is easier for the people using it to understand. From a real-world perspective, neither model is objectively better than the other. BKT has better applications in classroom dashboards, where teachers can use the easy-to-understand parameters and probabilities that the model provides to interpret where a student may be slipping and what concepts need to be re-taught for that student. However, for educational tools like problem sets and practice tools like Alcumus, using a DKT model is significantly better because it’s not necessarily important to understand why the model can accurately predict what difficulty of problem would be best for the student. Rather, it is more important to have a high accuracy in mastery prediction so that the model can provide engaging and challenging problems for the student that prompt improvement in scores. Both models are important when it comes to learning analytics, as BKT provides a more understandable view of student learning while DKT reflects real cross-dependencies between skills that may be causing weaknesses for students. In the future, more studies can be used to find models that implement both the accuracy of an RNN and the interpretability of statistical models to provide a model that is both accurate and interpretable. In conclusion, the experiment found that, though DKT is a much more accurate model in terms of correctness and mastery predictions, BKT is much easier to understand due to its interpretable parameters and statistical modeling. Though the experiment may be slightly erroneous due to older data (the ASSISTments 2017 dataset), it still provides enough evidence for the accuracy-interpretability tradeoff and demonstrates that DKT and BKT must be used in separate use-cases in education because of the tradeoff. **Abdelrahman, G., Wang, Y., & Karypis, G. (2022).** *Knowledge tracing: A survey.* International Journal of Artificial Intelligence in Education, 32(4), 1–40. **ASSISTments 2017 Data Mining Competition. (2017).** *ASSISTments 2017 dataset.* Worcester Polytechnic Institute & ASSISTments Foundation. [https://sites.google.com/view/assistmentsdatamining/home](https://sites.google.com/view/assistmentsdatamining/home) **He, X., Wang, Y., & Heffernan, N. (2026).** *Adaptive learning systems and their impact on learner mastery.* Journal of Educational Data Mining, 18(2), 55–72. **Majmudar, A. (2025).** *Interpretable knowledge tracing: Balancing transparency and predictive performance.* Proceedings of the Learning Analytics and Knowledge Conference, 112–124. **Shen, S., Chen, Y., & Li, X. (2024).** *A survey of knowledge tracing: Models, variants, and applications.* ACM Computing Surveys, 57(3), 1–38. [Comparing Accuracy and Interpretability in Knowledge Tracing Models for Mathematical Learning](https://pub.towardsai.net/comparing-accuracy-and-interpretability-in-knowledge-tracing-models-for-mathematical-learning-3faa7c449a03) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.