Research

When schools experience disruption, online professional development (PD) can provide educators with access to learning when traditional, face-to-face opportunities are limited. But simply making resources available is not enough. How do we know whether educators are actually using those resources, returning to them, or disengaging over time?

A recent study, “Measuring Educators’ Engagement with Professional Development During the Pandemic with Utilization, Retention, and Attrition Metrics,” explored this question using the EdHub Library, the online professional development platform developed by the University of Missouri’s Network for Educator Effectiveness. The study introduced three engagement metrics (utilization, retention, and attrition) derived from existing web analytics to examine educators’ engagement during the COVID-19 pandemic.

Looking Beyond Traditional Web Analytics

Online platforms generate a great deal of data, but measures such as pageviews do not necessarily tell us whether educators are meaningfully returning to professional learning over time.

The study therefore adapted concepts from learning analytics and other fields to create three practical measures:

• Utilization measures the extent to which educators access PD resources during a particular period.
• Retention captures the extent to which educators return to the platform and continue engaging with professional learning.
• Attrition identifies educators who stop engaging with PD resources over a specified period.

Together, these measures provide a more complete picture of engagement and disengagement than a single measure such as time on page or total users. The metrics were calculated monthly on a scale from 0% to 100%, allowing engagement to be tracked as conditions changed throughout the pandemic.

What Happened During COVID-19?

The patterns reflected important moments in the school calendar and the pandemic itself. Engagement peaked in March 2020, at the onset of the pandemic, and again around September 2020 and October 2020, near the beginning of the academic year.

Across the study period, EdHub recorded 24,147 unique users, divided almost evenly between 12,076 new and 12,071 returning users. This balance suggests the platform continued attracting new educators while also bringing existing users back during a period of significant educational disruption.

The newly developed metrics revealed additional patterns. Monthly utilization ranged from 3.49% to 17.15%, while retention ranged from 46.47% to 54.79%. Retention exceeded 50% in 8 of the 10 months examined. Attrition ranged from 3% to 15%, with the lowest rates occurring in August and September as educators prepared for the new school year.

These patterns show educator engagement with online PD is dynamic. Professional learning needs can increase or decrease depending on the school calendar, emerging instructional challenges, and external events.

Why Metrics Matter During Emergencies

COVID-19 demonstrated how quickly educators’ professional learning needs can change. During an emergency, PD providers need ways to recognize those changes rather than waiting until the crisis has passed to evaluate participation.

Utilization, retention, and attrition can function as practical indicators of the health of an online professional learning environment. Rising utilization can signal increased demand. Strong retention can indicate educators are continuing to find value in available resources. Increasing attrition, meanwhile, may signal a need to investigate whether content, accessibility, timing, or other factors are limiting continued participation.

For organizations such as NEE, this means that online PD analytics can become part of emergency preparedness, not simply program evaluation. Engagement data can help organizations determine when educators need additional support and how professional learning resources might be adjusted as circumstances change.

Practical Implications for PD Providers

Several practical lessons emerge from the study:

• Track engagement over time, not just total visits. Utilization, retention, and attrition reveal different dimensions of educator participation.
• Use high-traffic periods strategically. New resources and timely PD can be promoted when educators are most actively seeking professional learning.
• Pay attention to declining engagement. Low utilization or increasing attrition can help identify opportunities for outreach, collaboration, or changes in available resources.
• Align PD with the school calendar. Engagement patterns around the beginning and end of the school year suggest timing matters when developing and promoting resources.
• Build analytics into emergency planning. Engagement metrics can establish baselines that make it easier to recognize unusual changes in educator needs during future disruptions.

The broader lesson is that online PD platforms are more than repositories of professional learning materials. When paired with thoughtful learning analytics, they can help organizations understand how educators respond to changing circumstances and make more informed decisions about what support to provide and when to provide it. The study concludes that these metrics can help establish engagement and disengagement baselines during crises while supporting decisions about resource allocation, program improvement, and evolving professional learning needs.

About the Author: Dr. Javier Leung is an instructional designer and educational data scientist with the University of Missouri’s Network for Educator Effectiveness. His research focuses on learning analytics, educational data mining, and learning design, with particular emphasis on understanding how educators engage with online professional development. He applies data science and machine learning methods to examine patterns in educator behavior, evaluate digital learning environments, and translate educational data into practical insights for improving professional learning. His research explores how data-informed approaches can help educational organizations better understand educator needs, strengthen online PD, and support decision-making during both routine and disruptive educational conditions.