Research

Researchers examined whether students were able to distinguish among multiple dimensions of teaching on the NEE Student Survey. Findings indicate students were able to do so and that student ratings reflect a reliable measure of six constructs of teacher effectiveness. To access a link to the full study, click the blue “Source” button on the right side of this page.

Purpose of the Study

This study examines the validity and reliability of the NEE Student Survey. It aims to answer whether students can distinguish among multiple dimensions of teaching to understand whether the ratings measured by the survey capture the dimensions of teacher effectiveness that are intended.

Background of the Study

Student evaluation of teaching has been widely used at the post-secondary level for decades, but not as extensively in K-12 schools. However, teacher evaluation systems across the United States – including the Network for Educator Effectiveness — are increasingly including student surveys as one source of data.

Student surveys have some distinct advantages over other data sources for teacher evaluation. They uniquely provide the perspective of the learners who participate in the instruction. They are low cost, ranging from 1/5^th to 1/20^th the cost of classroom observations. And they provide immediate, actionable feedback if the survey instrument is detailed and well-designed.

Despite these advantages, teachers remain skeptical about youths’ ability to distinguish between effective and ineffective teaching and worry that ratings may simply reflect a teacher’s popularity or how easy the student finds the class subject.

About the NEE Student Survey

The NEE Student Survey is used to evaluate approximately 13,000 teachers by over 250,000 students in roughly 275 districts annually across Missouri in the United States. The survey’s framework is based on the Interstate New Teachers Assessment and Support Consortium (InTASC) standards (Council of Chief State School Officers, 2013).

The survey focuses on six InTASC standards that are readily observable by students in the classroom:

• Uses content knowledge and perspectives aligned with appropriate instruction
• Understands and encourages student learning, growth, and development
• Teaches for critical thinking
• Creates a positive classroom environment
• Uses effective communication
• Uses student assessment data to analyze and modify instruction

Each standard is comprised of two to five more detailed teaching practices for a total of 24 teaching practices. For each of the 24 teaching practices, the survey includes three to six items for a final bank of 96 items. Each item asks students to rate their teacher on a 4-point scale (0 = Not true, 1 = Sort of true, 2 = True, 3 = Very true). Each item asks students to focus on the teacher and not other students. There are also three validity items meant to identify careless responders.

The Research

This study applies multilevel modeling to investigate how students use the NEE Student Survey to evaluate teacher effectiveness. Multilevel confirmatory factor analysis (MCFA) was used to correctly estimate the standard errors for nested data and investigate the factor structure of the survey at individual student and teacher levels.

The sample included 5,178 student surveys evaluating 377 teachers. Among the teachers evaluated, the majority were high school teachers (60%), followed by middle school (26%), elementary school (11%), and teachers in other types of school settings (3%). Each teacher in the sample received an average of 15 student surveys.

Findings

• 4^th to 12^th graders’ perception of teacher effectiveness is consistent with the theory-based structure of the survey.
• The 24 teaching practices included on the NEE Student Survey are good indicators of their respective latent construct.
• Students’ ratings for the six theory-based factors reflect a sufficient portion of variability between teachers, and the aggregate student ratings reflect a reliable measure of the six theory-based teacher effectiveness constructs.

Reflection

These findings are critical for supporting the use and interpretation of NEE Student Survey results to direct teaching improvement efforts. It would be appropriate to use aggregate student ratings to reflect a teacher’s effectiveness as defined by the six InTASC-based standards. These results contribute to the growing literature that student ratings of teacher effectiveness are reliable for elementary, middle, and high school students and add credence to their use as a means of obtaining valuable information about teacher effectiveness.

Researchers have identified three concerns as districts adopt student surveys for teacher evaluation – that they measure what matters, are able to differentiate among teachers, and that only validated, well-developed measures are used. These results support the notion that student surveys designed around a research-based framework, such as the InTASC standards, may allow districts to address these concerns.