#309: Reexamining the Measurement of Metacognitive Regulation in Online STEM Learning: Validity and Fairness Evidence

This study reexamines the measurement of metacognitive regulation in online STEM learning in the context of broad access institutions serving more diverse student populations than the selective contexts where these measures were originally developed. Using data from 1,517 undergraduates enrolled in gateway STEM courses, we analyzed three regulation-of-cognition subscales from the Metacognitive Awareness Inventory: Comprehension Monitoring, Debugging, and Evaluation. We used exploratory factor analysis, item response theory, and differential item functioning (DIF) to evaluate dimensionality, reliability, and measurement invariance. Results supported a three-factor theoretical model, though a one-dimensional model provided better parsimony and fit. Items showed moderate discrimination and difficulty, with debugging items being harder to endorse. Measurement invariance was maintained across time, gender, and first-generation status, with minor DIF across students from different racial and ethnic backgrounds.