#319: At the Intersection of Cognitive Engagement and Critical Thinking in AI-Assisted Learning

This study examined how cognitive engagement levels, defined by the ICAP framework, relate to critical thinking in student-AI interactions. Analyzing 784 student questions submitted to a GPT-4-powered chatbot in an introductory biology course, we coded each question for ICAP level and applied Newman et al.’s (1995) critical-thinking indicators. ANOVA results revealed significant differences across ICAP categories: Constructive questions yielded the highest critical-thinking ratios, followed by Active and Passive. However, Constructive questions were mostly clarifying or confirmatory, rarely exhibiting evaluative reasoning. These findings suggest that higher ICAP engagement predicts higher critical thinking, but students often remain at explanatory rather than evaluative levels when interacting with LLMs. The findings shed light on the need for instructional scaffolds that promote both cognitive engagement and epistemic vigilance.