Ricochet is the best place on the internet to discuss the issues of the day, either through commenting on posts or writing your own for our active and dynamic community in a fully moderated environment. In addition, the Ricochet Audio Network offers over 50 original podcasts with new episodes released every day.
The New York Times Misrepresents Charter School Research
Yesterday, the New York Times ran a front-page story purporting to show that “betting big” on charters has produced “chaos” and a “glut of schools competing for some of the nation’s poorest students.” (One wonders how many of those low-income families are upset that they have “too many” options.). However, the article’s central claim about charter school performance rests on a distorted reading of the data.
The piece claims that “half the charters perform only as well, or worse than, Detroit’s traditional public schools.” This is a distortion of the research from Stanford University’s Center for Research on Education Outcomes (CREDO). Although the article actually cites this research – noting that it is “considered the gold standard of measurement by charter school supporters across the country” – it only does so to show that one particular charter chain in Detroit is low performing. (For the record, the “gold standard” is actually a random-assignment study. CREDO used a matching approach, which is more like a silver standard. But I digress.) The NYT article fails to mention that the same study found that “on average, charter students in Michigan gain an additional two months of learning in reading and math over their [traditional public school] counterparts. The charter students in Detroit gain over three months per year more than their counterparts at traditional public schools.”
As shown in this table from page 44 of the CREDO report, nearly half of Detroit’s charter schools outperformed the city’s traditional district schools in reading and math scores, while only one percent of charter schools performed worse in reading and only seven percent performed worse in math.
Grouping the very few underperforming charters with the approximately half of schools that perform at roughly the same level as the district schools distorts the picture. It’s just as fair to say that more than nine out of ten Detroit charters performed as well or better than their district school counterparts. The most accurate description would note that about half of Detroit’s charters outperform their district school counterparts, about half perform roughly the same, and a very small number underperform.
According to CREDO’s 2015 nationwide study, 60 percent of charter schools outperform their district school competition in math and 51 percent outperform the district schools in reading. By contrast, the district schools outperform only 8 percent and 4 percent of Detroit charters in math and reading, respectively. The following two charts from pages 29 and 31 of the report show comparisons of charter school performance in various cities against “the alternative schooling options their students face” (i.e., the nearby district schools to which students would otherwise be assigned).
In other words, the best available research on Detroit’s charter school sector shows almost exactly the opposite of what the NYT piece portrayed. Indeed, as Professor Jay P. Greene of the University of Arkansas noted:
To claim that half the charters perform the same or worse than traditional public schools is a grotesque distortion of the study’s findings. […] [I]f the reporter cites that research to demonstrate that one charter management organization has sub-par performance, it is journalistic malpractice not to mention the positive overall results. And those positive overall results contradict the very foundation of the entire article.
The NYT reporter, Kate Zernike, took to Twitter to defend her reporting against Greene’s takedown, citing data from Excellent Schools Detroit. However, as Greene explained, those data do not allow for direct comparisons. Zernike is right that the data show the citywide averages in each sector, but looking at the averages is misleading.
The charter schools tend to be mission-based schools that open in the toughest areas and serve the most at-risk students. Comparing city-wide averages fails to take that into account. It would be like comparing the New England Patriots against a championship high school team and concluding that the teenagers are superior athletes because they scored more touchdowns per game.
The appropriate comparison is between the charters and the district schools that serve the same or similar student populations. That is what the CREDO study attempted to do by matching students with similar characteristics and initial test scores in each sector, then tracking and comparing them.
Zernike is still claiming that the CREDO study “does not consider Detroit[’s charter sector] stellar,” even though both the 2013 CREDO study of Michigan’s charter sector and the 2015 CREDO study of charters nationwide found that, on average, Detroit’s charter schools outperformed the district schools that their students would otherwise have attended. Indeed, one even called Detroit’s charter sector “a model to other communities.”
Zernike is simply wrong.
For an even more detailed critique of the article, read Tom Gantert of the Mackinac Center for Public Policy here.
This blog post was originally published at Cato-at-Liberty.
Published in Education
Even if this stat were true, what’s the problem?
The law of averages predicts that half the schools would naturally perform below the average. That a portion of that 50% perform as well as the average means they’re exceeding expectations!
Also, the fact that 50% perform above the average is nothing to sneeze at!
Your well-researched and authentic critique deserves a wider audience! Poor Zernike doesn’t think for him or herself and indeed seems like what Rhodes would label a useful idiot in this particular echo chamber. New York Times readers (especially the ones who like in NY) should know the truth. If you haven’t responded to the NYT, please do it.
To paraphrase Will Rogers, there are three types of lies: lies, damned lies and the New York Times.
Several years ago the science section of that disgraceful rag ran an article that claimed that recent research showed that men and women had strikingly similar math science abilities and that therefore any difference in the number of hard science PhDs must be the result of gender bias. Except the study they were referring to didn’t show that. It did show that the mean for men and women was indeed very similar. As one would expect, both groups demonstrated a normal distribution (i.e., a typical bell curve). The two salient differences were that the mean of the women’s curve was a little higher than (to the right of) the men’s curve and the men’s curve had fat tails. So assuming the study was valid, it demonstrated that, on average, women were slightly better at math than men, but that there are more math whizzes and idiots that are men than women. If math PhDs are drawn from the population at large at random, then this study would suggest that some kind of bias is jiggling the results.
(cont)
(cont)
But, as should be obvious, even to a NYT writer, math Phds are disproportionately drawn from the population outside of 2 standard deviations (on the high side only, of course), thus the relevant portion of the population are the ends of those tails, which were very thin for women and fat for men. Meaning you would expect men to dominate in the hard sciences, as they do.
So when you tell me the NYT has cited a scientific study to support the exact opposite of what the study actually says, then I say, dog bites man, there’s no story here.
The NYT, if they didn’t print lies it would be the sports daily.
Thanks for the analysis. Well done. As someone who was a long-time reader of the Times, it became apparent over the years that most of its reporters combined political bias and unfamiliarity with statistics, or even basic math, including ignorance of methodology issues. That enables them to be easily spun by liberal advocacy groups. Stories like this usually start with an leftist advocacy group approaching a Times reporter with a “story” and the reporter, because it aligns with their political views, swallowing it hook, line and sinker.
What about the criticism that Detroit public schools are so bad that parents (or a parent) pull(s) their children out of the public schools and enroll(s) them in charters, thereby skewing the mix of students vis-a-vis the public schools? That is, children with intact or involved parents are more likely to do better in school and therefore by taking them out of public schools and putting them in charter schools, some, all, or most of the best students may no longer be in public schools and instead will be enrolled in charters. Does the study account for this possibility (I tend to think, given how inadequate and corrupt the Detroit public schools are, it’s more likely a probability)?
Good question. That would be an issue if we were comparing averages, as the NYT reporter did. But this study used a method that avoids that trap.
The CREDO study is not random-assignment (i.e., gold standard), but it uses a matching method that is second best. The matched students have nearly identical test scores and demographic backgrounds initially, so we can say with some degree of confidence that the difference in outcomes stems from their schooling.