Mark Dynarski of Brookings has a post arguing that the teacher evaluation reforms of the Obama era, and in particular their emphasis on using more frequent teacher observations and complex teacher observation rubrics(like the Danielson framework) have been a pointless waste of time and money. His argument can be boiled down as follows:
a) 12th grade NAEP scores not only haven’t improved, they’ve actually declined a bit, at least in reading.
b) Even if the new observation systems have gone from marking almost nobody ineffective to marking a few percent ineffective (and introduced a “highly effective” category that a smaller percentage get), it can’t be that the large majority of teachers are effective when scores aren’t going up.
This is kinda dumb. There have been huge compositional shifts in American schools, with the majority of American students now eligible for free or reduced price lunch, i.e., low income. The racial composition of American students has also shifted dramatically, with students from low scoring groups making up a larger portion of the under-18 population than ever before. And the changes in the composition of 12th graders have been especially large, since low-performing high school students, who used to drop out, are now more likely to persevere.
The Official Spotted Toad line on test scores is that you’re unlikely to change them that much by changing school practices, and you’re especially unlikely to change the differences between groups all that much by changing school practices. This doesn’t mean that making schools better isn’t worthwhile, or that test scores are uninformative, just that at the level of the whole country test scores are telling you much more about changes in composition of the student population than about changes in schools. (The same goes for comparisons between countries.)
But what about teacher observations? Are they, as Dynarski argues, totally useless? Well, if your goal is to eliminate differences in test scores by income and group: yes, they are probably useless. But if the question is- is it a good idea to give teachers more feedback than they were getting, I think the answer is yes. Teacher observations made at least as much sense as the other three legs of the Obama-era ed reform stool (Value Added Modeling, more charter schools, and much harder tests.)
I can make the argument by pointing out that the evidence for the validity of observation, really is decent enough. But I think the bigger issue, once you let go of the idea that you’re going to make every kid ace the end-of-year test, is that observations are in keeping with what most of us want schools to be, which is a place where we don’t feel ashamed to send our kids and where we as adults can without misery spend some time. If there’s one strange thing about the way we bring up kids, it is that we stick them all in a room together with just one adult and expect them suddenly to adopt adult attitudes and interests. Isolation as a teacher is a more present danger than you might anticipate when you start; the sense that you are the sole adult on Pinocchio’s Island of Naughty Children arrives faster than you expect. Many teachers would not need formal, required observations to make their classrooms open to instead of closed off from the world, but even the classrooms of the “best” teachers go sour more often than many expect, and, to drop my habitual pessimism for a moment, my own sense from visiting urban schools over the last eighteen years is that there has been a real change in this respect, with fewer teachers viewing their room as a private demesnes to be defended at all costs, more comfortable discussing their work relatively dispassionately and openly.
This shift predated Obama-era reforms, and cannot be attributed mainly to them. No doubt much money has been wasted on observation systems, because money is always being wasted, it’s what money does. But the open classroom door, or the door that can be opened without fear of what lies behind, is not an incidental or unimportant part of what we want from schools, the Vale of Shadows behind which the Platonic form of test score truth lies concealed. It is instead the main event itself. What you see is what you get.