#77: Developing an Approach for Evaluating Designs of Generative AI-Supported Learning

While literature observes a surge in Generative AI (GenAI)-supported learning designs, these designs are often evaluated narrowly based on learning outcomes, offering limited insight into how GenAI contributes to pedagogical designs and supports learning processes. This paper addresses this gap by proposing a holistic evaluation framework that encompasses three interrelated interactions: (1) interactions between learning designers and GenAI in shaping design embodiment, (2) interactions between GenAI and learners in mediating learning processes, and (3) interactions facilitating Explainable AI, focusing on how learning designers and learners understand, validate and improve the responses of GenAI. Grounded in Conjecture Mapping (Sandoval, 2014), the proposed framework organizes the evaluation of these interactions to inform how learning designs lead to intended learning outcomes. The framework provides a systematic means for evaluating and improving GenAI–supported learning designs beyond learning outcomes alone.