“Schools that Work”

A few different readers (and friends and family members) have sent me this David Leonhardt essay about the efficacy of Boston charter schools, based on yet another Josh Angrist study demonstrating this efficacy, and asking what I think about it.

I think the Boston charters are good for three reasons:

• Lots and lots of smart people in Boston to staff them
• Competition from a relatively high-functioning urban district.
• Liberal/union politics that constrain them from growing and expanding too fast.

This is more-or-less what I tried to argue here. Freddie DeBoer, whom I don’t exactly always agree with, made a similar argument about Success Academies.

The question, then, is whether Angrist’s findings generalize to the rest of the country apart from Boston, and to charter schools’ ability to ameliorate inequality in different settings or in places where the students and parents haven’t deliberately selected into an extra-intensive schooling system.

There are at least two recent studies that suggest that the kinds of intensive interventions these Boston charter schools employ work, to a degree, in settings without selection.

1. Angrist (and a different set of colleagues) find recently that students who were in low-performing district schools that were taken over by charter schools do better than similar students in schools that weren’t taken over. The study was done in Boston and New Orleans. Boston is Boston.  As for New Orleans…I think when you have a really ghastly legacy system, and a lot of money and new blood coming in (see above) you can make a big difference. I had some friends who taught in New Orleans pre-Katrina, and it sounded like a much worse disaster than the crappy school I taught at in the Bronx- lots of teachers and administrators smacking the kids around, crazy open classrooms where you’re basically in a warehouse-style room with five different classes of elementary kids that you’re trying to keep from running from one group to another, and so on. If the counterfactual is crappy enough, and you have enough resources coming in, you can make a big difference.
2. Roland Fryer of Harvard worked with the Houston schools to randomly assign a group of low-performing elementary schools to use a set of charter-school methods, and got quite good results, particularly from an intensive (and expensive) tutoring model borrowed from the Match charter school that Leonhardt discussed.

There remains something weird about us adopting such an intense mode of schooling as the baseline, but perhaps it makes sense in a world where kids don’t have a lot of freedom to play and socialize when they come home.  My goal for schools is that they are pleasant and structured and challenging places, with enough going on that kids can grow up in them, but that they are not a substitute for growing up, especially given that none of us have any clue what we are supposed to be preparing kids for.

There are other barriers to expansion as well, of course, quite aside from the widespread corruption and political dysfunction that has resulted in places like Philly or Newark  or Tennessee when philanthropists and politicians have gotten together to speed the plow of education reform forward, and quite apart from the stubborn fact that nationally representative studies of charter schools show no positive impact at all.

When the Department of Education offered oodles of dough to do en masse the kind of district-to-charter conversions that Angrist found effective in Boston and New Orleans, they only got 30 takers nationwide, with no assurance of whether those 30 found much success. When Boston Public Schools adopted a stricter teacher evaluation system inspired by the ones charters use,  they ran into a civil rights landmine, with black, male, and older teachers all scoring much worse and more liable to dismissal under the new system.

There also is a question about whether the growth and flourishing of high-intensity charter schools required the mixture of revenue-rich municipalities and a glut of certified teachers that we’ve seen on-and-off over the last decade and a half. The glut of teachers, at any rate, looks likely to turn into a shortage quite soon, and with the exceptions of some high-flying places like Boston, I’m skeptical whether municipal and state finances will ever again be self-sustaining in most places, or if the end of educational localism and the imposition of a nationally managed school system will be the result not of a tyrannous federal government but of dead-broke schools.

One of Leonhardt’s more eyebrow-raising claims is that Match didn’t just reduce the achievement gap between black and white students, it eliminated it, on the 8th grade standardized tests at any rate. Education Realist might call this the “false god of elementary test scores,” citing the persistence of gaps on the SAT and other high-school level tests, and while I do think 8th grade test scores measure some important skills, it does seem like the dinosaurs might have gotten loose in Angrist’s particular study. Previous studies of highly effective charter schools like KIPP have suggested that we can close a decent portion of the achievement gap (25-30% maybe  ) with these intensive modes of instruction, which is great, but Fryer and other folks have a tendency to talk about this as a “per year” effect, when in practice most studies have indicated you can get this size of effect in a year…and then it either fades out afterwards or you can maintain it by maintaining similarly intense schooling. The latest KIPP randomized trial, that included high schools for the first time, suggested large effects for new students to KIPP but no measurable additional marginal effect for kids who had already gone to a KIPP middle school.

Of course, you could conceivably have some Heckman-style noncognitive benefits that do accumulate over time, even if cognitive benefits are not cumulative, or some substitution between cognitive and noncognitive abilities as kids age. That is, a kid who comes in further along as a reader or just more adjusted to school and understands what’s going on can relax a little bit more and enjoy the day-to-day, while a kid who is really behind, even if he catches up substantially (and ends up where he was going to in the end regardless), has a less pleasant time while doing it, with the result of less social adjustment to school, which Heckman measures as non-cognitive skills. As much as I’m skeptical of what Heckman has been saying recently about genetics and epigenetics, I don’t think this kind of substitution is implausible. I like Scott Alexander’s more contrarian way of saying the same thing:

If I wanted to be very mean (and I do!) I could even say that all kindergarten is a neurological insult that destroys later life prospects because of forcing students to overclock their young brains concentrating on boring things, but good teachers can make this less bad than it might otherwise be by making their classes a little more enjoyable.

There also is the interesting-to-me idea that high-intensity charters might be better for some kids and worse for others. Middle class kids, on average, often have negative impacts from attending charter schools, for example in CREDO’s national study:

Perhaps, then, this is not a matter of better or worse, but of better or worse for whom, and in particular better or worse for different styles and levels of general cognitive ability or (in my formulation), different predispositions towards abstract and individually-articulated versus concrete and socially-embedded thought. This is analogous to the famous argument that Lisa Delpit made thirty years ago, arguing against progressive education, albeit in her case in terms of languages and cultures of power rather than in terms of cognitive dispositions and abilities:

 

In some ways, it may not be important whether Delpit or I am right or wrong (I for one will say that she is at least largely right, especially when she talks about language), or indeed if the dispositions of culture or language or thought are socially constructed rather than or in addition to being biologically embedded. As Gabriel Rossman has noted, the goal of studying sociology should be that you don’t think that just because something is a social construction, you can click your ruby slippers together and make it go away. Even with the best-executed of charter schools guiding kids’ sails (as they set keels to breakers, unto the godly sea), the question remains how long they will need that same kind of structure and support to continue. KIPP has long put almost as much emphasis on finding ways to support their graduates as their current students, and at least some administrators at quite selective institutions think that similar support needs to continue at least through the beginning of college for many even high-achieving black and Latino kids. My own speculation is that at any level, academic gaps tend to magnify themselves through social sorting, which is part of the reason HBCUs tend to have continued success in spite of paltry funding and support.

The broad changes in demographics in the American school system will continue to require changes and accommodation from schools; currently it at least appears that the consensus movement of left-liberal thought is away from high-stakes education reform and towards forced integration (or more bloodlessly, “diversity”) as a solution to educational inequity. My read of the relevant studies is that diversity per se has a bit less evidentiary support than the charter schools Leonhardt is ballyhooing, but (more cynically) moving low-income black and Hispanic families from high-growth cities to “middle-class” suburban and rural areas, as Affirmatively Furthering Fair Housing and other current Obama Administration initiatives seek to do, has the advantage of distributing future Democratic votes more evenly and allowing for Democratic-leaning real estate developers to make more money in the bargain. This, combined with a broadening disdain for the “savior mentality” that energized TFA and reform-minded charter schools in the past,  says to me that the lumbering beast of bien-pensant opinion will continue to lumber away from education reform.

Given these changes in the political winds, Leonhardt can at least be credited with sticking to his read of the evidence, something that to his credit he has done in other areas where crystallized left-liberal opinion also does not agree with his. My read of the evidence on charter schools disagrees with his- Boston charter schools are less “one part of education reform,” than a fairly small group of schools in a fairly small (and quite unusually wealthy and well-educated) city, a group of schools that rose to success and prominence in a particular time and whose success can only be replicated contingently and partially rather than systematically nation-wide. But even contingent and particularized success remains interesting, worth considering, and possibly inspiring to those who must find a way to show up in their particular school building on another Monday with a partial measure of curiosity and joy.