Date Published: January 2022

Leung, J. (2022). Evaluating Teacher Access to Online Professional Development: Establishing Access Patterns from User and Server Analytics. AECT Selected Papers On Practice of Educational Communications and Technology, Volume 2. https://members.aect.org/pdf/Proceedings/proceedings21/2021i/21_17.pdf

Source

“After analyzing 164,772 web analytics transactions from May 2018 -May 2021, the study provides internal stakeholders with recommendations for improving generic keyword searches with curated lists of resources and optimizing current multimedia resources to reduce load times.” (Leung, 2022)

Purpose of the Study

The EdHub Library is the online professional development platform of the Network for Educator Effectiveness. EdHub provides teachers with 24/7 access to more than 500 online resources. Like other online resources, EdHub has experienced increased web traffic since the beginning of the pandemic.

The purpose of the study is to evaluate and improve access to online resources within EdHub by analyzing user and server web analytics requests.

This study seeks to identify the most-accessed resources and develop user and server access patterns in online teacher professional development resources. The significance of this study involves providing improved access to resources, especially to rural Missouri school districts. The study explores the following research questions:

  1. Which resources do users access the most based on Page Depth and Time On Page?
  2. Which resources have the highest Server Response Time, Document Content Loaded Time, and Page Loaded Time?
  3. What access patterns exist for users and server document requests?

Background of the Study

Online teacher professional development programs have gained popularity over the past 10 years, especially during the Covid-19 pandemic. There is, however, a lack of empirical evidence that delves into online professional development, its usage, and its effectiveness.

While there are ongoing studies on program design and effectiveness, there is especially a lack of evidence on the technical and learner interactions with online professional development platforms like EdHub.

The Research

From May 2018 to May 2021, EdHub materials were accessed 164,772 times. From those transactions, an exploratory analysis was conducted to identify the most-accessed resources and find navigation patterns. The study used analytics to measure Page Depth, Time on Page, Server Response Time, Page Loaded Time, and Document Loaded Time.

The study also used the Apriori algorithm to help establish patterns between antecedent and consequent components of user and server resource access. The user and server resource access are based on support, confidence, and lift measures.

Findings

Through the analysis, NEE and EdHub took into consideration seven best practices for making resources more accessible:

  1. Avoid unnecessary redirects of resources
  2. Reduce the size of files
  3. Design for a variety of interaction methods (mobile and desktop) and internet speeds
  4. Use adequate text size with minimal Cascading Style Sheets
  5. Implement high-quality graphics with a smaller footprint
  6. Use cookies when necessary
  7. Reduce the size of HTML pages to avoid rendering errors

In EdHub, videos of teaching best practices and of walkthrough examples of classroom observations are accessed frequently. To make these more accessible, NEE has implemented progressive download to transfer media between the server and client before the user can render the full video. This helps NEE stay away from the costs of a streaming service while also providing quick downloads of the media.

NEE’s current dataset could not determine whether the most common searches – such as “real world” and “word problems” – provided a successful hit for the user. This informs us of future studies concerning user behavior when using the search engine, as well as curating professional development materials for the most common search terms.

Reflection

As NEE says frequently, data is the beginning of the conversation. We use the same approach in our internal decision making. This research study shows the depth and ways in which we can access user experiences to inform the next steps. EdHub plays a critical role in what NEE is; it provides quick and effective professional development modules available to each user 24/7.

NEE and EdHub are investigating best paths forward to connect the most common keyword search terms with curated professional development modules to make the user experience more friendly. We also continue to seek ways to optimize the multimedia experience of our users to reduce load times and provide resources to users even more quickly.