#359: Design and Perceptions of a Science Conversation-based Assessment with AI Agents

Conversation-based assessments (CBAs) incorporate conversational agents into computer-based tasks to elicit evidence about learners’ knowledge and skills in a domain. We describe the design of a science conversation-based assessment incorporating AI agents that engages high school students in a scenario related to storm formation and storm prediction. We further discuss results from a usability study in which students completed the CBA task and subsequently responded to interview questions about their perceptions related to the design and usability of the task. Students generally had positive experiences with the task and its design, though challenges related to information clarity and AI interaction authenticity were noted.