<p>Merit pay is one of those ideas which seems logical but which isn't. Here's why: there will be no correlation from year to year of performance so we will reward this group one year and this group the next. </p>
<p>We know this from the work of people like Daniel Kahneman, one of the pioneers in cognitive science and behavioral economics. One of my favorite things he has written and talked about is his experience early on with Wall Street. He was asked to review compensation at a securities firm and was given their performance rating system data. When he went through it, he realized they were rewarding based on performance that the data said didn't correlate to much of anything other than that it happened. By that, I mean performance varied enough year to year that the changes were chance from year to year. This meant some guy got a bonus one year and a pat on the back for doing great when it just luck and the bad year he had 2 years ago was just luck, etc. Note this assumes the people were all performing with at least the basic skill of the people they hired. Idiots would either get fired or not be hired, just as a guy who outperformed year after year would stand out. </p>
<p>If we assume that teachers are roughly similar - and that can be very roughly similar, meaning a fairly significant variation - and we apply those teachers to groups of kids who vary year to year, then you get garbage as a performance rating. We can identify the idiots and we may be able to identify the stars, though that's less likely given the variability of the kids year to year, but we can't say much useful about the vast number who aren't idiots or stars. </p>
<p>As a note, when MA started the MCAS assessments each school in our town made a big deal of how it ranked versus the others. The data kept piling up. You can see over time that the performance varies with school x better then school y then school z and whatever. It's essentially random. Not the overall level of our district performance, which shows we're very good, but the performance within that "very good". And if you look at our town and other high performing towns, you see that good schools reflect money and aspiration. I once did a quick run through of how much you could attribute the difference in our performance to one of the much more homogenous wealthy districts in the burbs and found it was quite possibly - see, it's a guess - more than the measured gap. By that, I mean the data suggested to me that we were over-performing relative to expectations given our diversity of race, ethnicity and income, but I didn't even try to adjust for the self-selection bias inherent in people choosing to be in this school district. I thought this might be a nice proxy for the power of aspiration but only in passing.</p>
<p>To me, the best way to evaluate teachers is through the principal, but that assumes the principal is competent and that mean the best way to evaluate teachers is by having better principals. Sometimes principals will screw up a school. A recent case was noted in our paper about a nearby school which spiraled down after the principal got rid of the teachers who didn't support his agenda. This makes the teachers union hesitant to give principals more power, but they are the managers in our system. I suggest we focus on getting better managers and a better system that supports these managers, including relationships across schools - which are "departments" - because a teacher who has issues in one place may thrive in another and a good organization recognizes that.</p>
<p>I'm saddened we focus on the kind of rough metrics all intelligent industry has largely abandoned. We're applying the kind of crude measurements used to evaluate sales staff at Best Buy (how many warranties you've sold) or at a market (how many transactions per hour).</p>
