Submitted by Marianna Shershneva on

Cook DA, Stephenson CR. Validation of the Learner Engagement Instrument for continuing professional development. Acad Med. 2024;99(9):1024-1031. doi: 10.1097/ACM.0000000000005749.

Abstract

Purpose: Learner engagement is the energy learners exert to remain focused and motivated to learn. The Learner Engagement Instrument (LEI) was developed to measure learner engagement in a short continuing professional development (CPD) activity. The authors validated LEI scores using validity evidence of internal structure and relationships with other variables. Method: Participants attended 1 of 4 CPD courses (1 in-person, 2 online livestreamed, and 1 either in-person or livestreamed) in 2018, 2020, 2021, and 2022. Confirmatory factor analysis was used to examine model fit for several alternative structural models, separately for each course. The authors also conducted a generalizability study to estimate score reliability. Associations were evaluated between LEI scores and Continuing Medical Education Teaching Effectiveness (CMETE) scores and participant demographics. Statistical methods accounted for repeated measures by participants. Results: Four hundred fifteen unique participants attended 203 different CPD presentations and completed the LEI 11,567 times. The originally hypothesized 4-domain model of learner engagement (domains: emotional, behavioral, cognitive in-class, cognitive out-of-class) demonstrated best model fit in all 4 courses, with comparative fit index ≥ 0.99, standardized root mean square residual ≤ 0.031, and root mean square error of approximation ≤ 0.047. The reliability for overall scores and domain scores were all acceptable (50-rater G-coefficient ≥ 0.74) except for the cognitive in-class domain (50-rater G-coefficient of 0.55 to 0.66). Findings were similar for both in-person and online delivery modalities. Correlation of LEI scores with teaching effectiveness was confirmed (rho=0.58), and a small correlation was found with participant age (rho=0.19); other associations were small and not statistically significant. Using these findings, we generated a shortened 4-item instrument, the LEI Short Form. Conclusions: This study confirms a 4-domain model of learner engagement and provides validity evidence that supports using LEI scores to measure learner engagement in both in-person and livestreamed CPD activities.

Questions

  1. Why is measuring learner engagement in in-person and livestreamed educational activities important to medical educators? How would you define an engaged learner in the activities you facilitate?
  2. The article described several tools, including the Learner Engagement Instrument (LEI), the Continuing Medical Education Teaching Effectiveness (CMETE) tool, a 2-item scale from the Maslach Burnout Inventory, and a 2-item digital savviness scale. Have you used any of these tools in your educational or research practice? What insights did this article provide about LEI?
  3. How relevant do you find the authors' proposed 4-item LEI for your clinical teaching practice?

Comments

cakorib cakorib replied on

 Measuring engagement across in-person and livestreamed CPD matters because we keep assuming attention equals learning, and the modality can hide disengagement in different ways. An engaged learner in my sessions is asking targeted questions, attempting the practice tasks, and circling back to apply the content rather than just rating the speaker highly. I haven’t used the LEI before, but I have used brief burnout items and I appreciated the validity work here, especially the consistent 4-domain fit and the weaker cognitive in-class reliability. The 4-item short form feels practical for routine feedback, though I’d still want a fuller instrument when evaluating major curriculum changes. Also, our course eval inbox keeps getting junk links such as https://crazytimegame.org/ and https://xxxtremeroulette.com/ which makes me extra careful about keeping measurement tools clean and clearly educational .

cakorib cakorib replied on

Ethan Master replied on

Measuring learner engagement across both in-person and livestreamed CPD is essential because visible attendance does not always reflect meaningful learning. For me, an engaged learner demonstrates curiosity through focused questions, active participation, reflection, and clear intent to apply new knowledge in practice. I found the validated 4-domain LEI especially valuable because it captures engagement beyond surface attention. The proposed 4-item short form seems highly practical for routine clinical teaching feedback while still preserving meaningful insight into learner experience. Best SEO Companies USA | How SEO Helps Roofing Companies Win More Local Leads

Ethan Master replied on

Ethan Master replied on

Measuring learner engagement in both face-to-face and livestreamed CPD is highly valuable because participation alone does not guarantee attention, reflection, or future behavior change. In my teaching, an engaged learner is someone who listens actively, contributes relevant questions, connects ideas to prior clinical experience, and shows willingness to apply insights after the session. I found the validated 4-domain LEI model especially useful because it recognizes emotional and cognitive dimensions that standard satisfaction surveys often miss, making the short 4-item version very relevant for practical teaching feedback. AI Security Certification | Modern Security

Ethan Master replied on

will smith replied on

This article provides strong evidence that learner engagement can be measured in a structured and reliable way, especially through the validated LEI framework. The four-domain model (emotional, behavioral, and cognitive aspects) highlights how engagement goes beyond simple attention and connects closely with teaching effectiveness. It’s interesting how even small improvements in engagement measurement can lead to better learning design. In a way, it reminds me of agar online, where staying actively involved and adapting your strategy makes all the difference—just like learners who are truly engaged tend to achieve better outcomes.

will smith replied on