A Cognitive Lab Study of Student AI Chatbot Use in Online Coursework

Abstract

This cognitive lab study examined how secondary students in Grades 7-12 interacted with an AI-enabled chatbot while completing online course activities. Using chat logs, facilitator observations, post-session reflections, and survey responses, we analyzed spontaneous and lightly seeded interactions to evaluate whether the chatbot supported its intended theory of action: helping students continue learning without simply completing work for them. Findings highlight the importance of typing fluency, answer-boundary handling, safety and privacy response patterns, and student-facing feedback mechanisms. This study illustrates how cognitive lab evidence can inform ongoing validation.

- Anthology ID:
- 2026.aimecon-wip.1
- Volume:
- [Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress](https://aclanthology.org/volumes/2026.aimecon-wip/)
- Month:

• October
• Year:
• 2026
• Address:
• Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
• Editors:
• Joshua Wilson ,Christopher Ormerod ,Magdalen Beiting-Parrish

- Venue:
- [AIME-Con](https://aclanthology.org/venues/aimecon/)
- SIG:
- Publisher:
- National Council on Measurement in Education (NCME)
- Note:
- Pages:

• 1–8

- Language:
- URL:
- [https://aclanthology.org/2026.aimecon-wip.1/](https://aclanthology.org/2026.aimecon-wip.1/)
- DOI:
- Cite (ACL):

• Jinah Choi, Sonya Powers, Farzan Karimi-Malekabadi, and Michelle Barrett. 2026. A Cognitive Lab Study of Student AI Chatbot Use in Online Coursework . InProceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress , pages 1–8, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).

- Cite (Informal):
- [A Cognitive Lab Study of Student AI Chatbot Use in Online Coursework](https://aclanthology.org/2026.aimecon-wip.1/) (Choi et al., AIME-Con 2026)
- PDF:
- [https://aclanthology.org/2026.aimecon-wip.1.pdf](https://aclanthology.org/2026.aimecon-wip.1.pdf)