fivethirtyeight.com: The Science Of Grading Teachers Gets High Marks

[ucbalumnus]

http://fivethirtyeight.com/features/the-science-of-grading-teachers-gets-high-marks/ is a discussion about teacher evaluation and attempts to eliminate confounding factors from the evaluations.

@Andrew Flowers at fivethirtyeight.com

At the center of this debate are evaluation models that try to isolate the educational value added by individual teachers, as measured by their students’ standardized-test scores relative to what one would expect given those students’ prior scores. The hard part, as Friedman says, is to “make sure that when you rate a teacher, that you actually rate what the teacher has done, and not whether they had a bunch of very poor or very rich students.”

[Ynotgo]

Oddly, the article fails to quote or attempt to refute the American Statistical Association, which stated,

Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Ranking teachers by their VAM scores can have unintended consequences that reduce quality.

https://www.amstat.org/policy/pdfs/ASA_VAM_Statement.pdf

There are so many holes in VAM, it just makes me angry that they use it. It would make me more angry if they used it in our district, but they don’t.

Here’s an example: Do you know which teachers get rated as the lowest-performing teachers in any state by VAM methodologies? It’s the teachers at the STEM and other high-achieving magnets. Why? Because their kids can’t show a year worth of growth on the state tests, because the tests are so easy for those kids that they have topped out the maximum score long before they graduate from that level of school. And, the tests are scaled so that if a kid goes from 0 wrong one year to -1 wrong, they drop a huge number of points (vs. -20 to -21 hardly affects the score). So, the teacher gets dinged because a kid was careless or a question was ambiguously worded for a really bright kid.

[shawnspencer]

I am curious to what people think of a periodic rating system based on feedback based on the students themselves. They are the ones actually in the classroom and I think a system where they can provide anonymous, detailed feedback on a teacher in key categories: how much I have learned during the year, how clear they were at explaining things, how engaging and innovative they are with making you think on a deeper level, and how helpful they were when you approached them would be effective.

It would obviously have to be controlled for ratings that are simply poor just because of a bad grade or a student that didn’t take the survey seriously, but I think it could provide a valuable insight into what goes on inside the classroom. It also could provide areas the teacher could improve on, especially if there is consistent feedback saying so. I think a rating system based on a standardized test system is silly, partially because it does not account for so many confounding variables, but also because some of the best and most inspiring teachers that I have learned the most from did not teach to the test, but engaged me to apply the knowledge learned outside the classroom. On the other hand, I have had some very poor teachers who were either not engaged, spent most of class talking about extraneous material (ex: their cat), or expected students to memorize a textbook. That to me is not teaching

[ucbalumnus]

Student ratings of teachers can also have confounding factors. You might favor the teachers who give the best intellectual stimulation, go into the material in depth, etc., but there are other students who just want to make the minimum possible effort to make a desired grade in the course who may complain about the teachers whom you like.

[KKmama]

What do you do for school districts where many of the kids are hungry or have little sleep due to environment or no parental support? Teachers sometimes do not have a lot to work with.

My daughter who teaches in a tiny native village in AK is judged by the same standard as teachers in Fairbanks. Most of her students are extremely poor, deal with addiction issues in the home, have no expectation of ever doing anything other than remaining in the village and living a subsistence lifestyle. There is no incentive to learn higher math or US history. Or read a book for the fun of it.

The kids love and respect her and standardized test scores have improved greatly under her watch. But that is a rare exception for her district.

[MADad]

Student ratings of teachers are undependable. There are many kids whose favorite teachers are those who tell a great joke, entertain them, but demand very little of them–good grades for minimal work. These same kids would downgrade a teacher who makes them work, holds them accountable, and does not give away grades for little effort.

These may not be the school’s best students, but in a comprehensive high school, you’ve got the entire spectrum of achievers. I teach in a high achieving high school in a state with highly regarded schools, yet I am disappointed with the number of kids who have no interest in reading for pleasure, or doing anything that doesn’t lead directly to a grade.