#330: AI agents as a Refinement Tool for Operationalizing Coding Process in the Context of Textbook Analysis

Qualitative research often employs coding to analyze complex data, yet even with codebooks, the operational criteria guiding coders’ decisions frequently remain implicit. This gap limits transparency and comparability across studies. We explore how an AI agent (ChatGPT-5) can function as a refinement tool to surface and formalize tacit reasoning in coding. Using textbook analysis as a case, we applied the Cognitive Demand Level (CDL) framework to 47 fraction-related problems from Grade 3 mathematics textbooks in Iran and Korea. Through iterative engagement, ChatGPT-5’s discrepant coding prompted us to articulate, refine, and document operational criteria, evolving from an intuitive flowchart to a detailed criteria table. This process transformed implicit reasoning into explicit decision rules that can be scrutinized and tested by others. We highlight AI’s affordance as a collaborative refinement tool: not replacing human coders, but scaffolding transparency, comparability, and methodological rigor in qualitative research.