Ricochet is the best place on the internet to discuss the issues of the day, either through commenting on posts or writing your own for our active and dynamic community in a fully moderated environment. In addition, the Ricochet Audio Network offers over 50 original podcasts with new episodes released every day.
Education Spending and Results: An Idaho Story
So does spending more per student improve outcomes in Idaho? In a word, no.
I have a graduate-level background in statistics. I was asked by a local conservative think tank to see whether increased spending on secondary education on Idaho has a meaningful effect on student outcomes. Idaho periodically administers its own ISAT achievement tests to elementary and secondary students. The 10th grade science test from 2014 was the latest set of results that I could obtain with a broad sampling of students with the longest exposure to public schooling. Public schools in Idaho also submit their annual budgetary information on a standardized set of accounts, 2013 was the latest available. I extracted the operating portion of districts’ budgets (educational and overhead expenses) and scaled that by the average daily attendance (ADA) in each district, showing how much was being spent per student.
Using a statistical regression test, there is no significant positive effect of increasing spending on testing outcome, across a 3x range of expenditures. The cash might as well be burnt for all the good it’s doing, based on objective testing.
Some details: The ISAT yields one of four results: Below Basic (fail), Basic, Proficient, and Advanced. The chart above shows the total of all results better than fail, which should be a minimum expectation of performance by the educational system. However, I repeated the test using both the Proficient+Advanced total, and Advanced alone, and there were no positive effects on these higher level outcomes either.
I used a total of 99 school districts’ data. While Idaho has a strong charter school movement, that data was omitted to avoid an ‘apples and oranges’ comparison because they did not administer the same test. I also omitted a number of very small districts where overhead per student might negatively bias the outcome, and a few districts that are entirely on Native American reservations. I used only operating spending, dropping categories such as capital expenditures, bond repayment or food programs that are inconsistent across districts or have no immediate effect on students.
The highest spending district in Idaho was Blaine County, home of the blue state exclaves Sun Valley, Hailey, and Ketchum. Blaine spent $18,333 in operating expenses per student. The lowest spender was rural agricultural Jefferson County, at $5,189 per student. Jefferson’s pass rate on the 10th grade test was 85.4%. Spending 3.5 times as much, Blaine had a 87.5% pass rate in its affluent district. The most effective district in Idaho, in terms of pass rate per dollar spent, was the small agricultural community of Fruitland, which has an over 20% Hispanic population.
There may be ways to improve educational outcomes, but at least in Idaho, throwing money at the problem is not one of them.
Published in Education
How to advance knowledge is to post this in the Sunday newspaper and then follow it up with studies and essays and pieces every Sunday. Statistics are really hard to absorb all at once.
People need this kind of analysis to get smarter and they should have it. Usually the papers don’t provide it.
On a more pressing note, I’m dying to know what that $13,000 %30 dot means.
DEpressing is the word you’re looking for, I’m afraid. That’s a district called Plummer-Worley, that is 70% minority, most of it Native American. And yeah, those are the actual results, not an error. You can take a guess on what the totally on-the-rez districts would look like, and you’d be right.
Could you get in touch with the editor of the Boise Weekly and talk about the difficulty of Native American education and the failure of public education?
I don’t need details, but was it a multivariate analysis? I would think that a simple regression of score vs. spending would not be a proper analysis, as omitted variables could undermine the validity of the result. For example, if the high spending is concentrated in areas with bad demographics (from a testing standpoint), omitting the demographic variable(s) would give an inaccurate result.
The Boise Weekly is leftist rag and unlikely to want to engage with this. At any rate, I’m a number cruncher, not an educator. I can point out the problem, and that the unions’ & legislators’ throw-money-at-it approach has failed, but I’m short on solutions.
Locke On (Vie:
Maybe they will maybe they won’t. I ask that you call them up and talk about it. You never know unless you try.
To a more universal point, why do Native Americans test so low?
The #1 indicator of education achievement success in primary and secondary schools: parents who pay attention.
Excellent issue and question. I did present this as a univariate analysis for the sake of simplicity, but that was after doing a full up analysis, and I believe I have compensated for the omitted variables. Geek_level++:
An obvious issue is district size – small rural districts have much higher overhead per student, a subject of continual fighting between communities wanting control vs. state funders wanting efficiency. I did a plot of spending/ADA vs. district size, there was an obvious knee in the curve, and I omitted districts below that level.
Ethnic mix. This is highly significant. I can only get white %age, due to privacy issues in the dataset, but that is positively correlated with results. However, spending level (at least in Idaho) is not highly related, due to a patchwork of subsidies on one hand, and rich/poor areas on the other.
Poverty level. Also highly significant. This can be deduced from the dataset since it gives the number of students on food assistance. One of the reasons I dropped expenditures on those programs.
Local funding sources. Marginally significant. Turns out the willingness of the district’s citizens to vote supplemental levies and/or provide other purely local sources of funds is positively related to outcomes. Logically, this shows community support for the value of education, since it does NOT matter how it’s spent.
Once you’ve accounted for those, you are out of significant variables in the dataset, IOW funding level is not significant in either the univariate or full analysis.
BTW, those three significant independents account for only about 50% of variance (and a scree plot descends into noise just about as quickly). There’s a lot going on that’s not captured even in my larger dataset…
In Maryland’s Baltimore County the average spent per student is in the neighborhood of $18k and yet those schools are among the worst in the country.
Based on Charles Murray’s work in The Bell Curve, I suspect that IQ and parental socioeconomic status (SES) would help explain more of the variance. I realize that you probably don’t have such information in your dataset. Poverty level would be something of a proxy for SES, and %white would probably be something of a proxy for IQ.
I don’t know Idaho well enough to comment on urban/rural differences. I would generally expect urban areas to have higher costs, due to higher land values and cost of living (and thus the need to pay teachers more). This might partially offset the district size issue.
I’m promoting this post to the Main Feed for 4PM ET. Comments won’t work until then.
@lockeon:
I’d guess that most of the Educrats would say that the ISAT doesn’t really measure student learning or “critical thinking” (or whatever they want to claim as the “right” measure for student outcomes) so your pretty little analysis means nothing…give us more money!!!
So, do you have any stats on how the ISAT correlates with later student achievement (e.g., admission to “top” universities)?
Unfortunately, objectivity doesn’t matter. It feels good to spend more money on education so the majority of people are going to support it.
No, the tests haven’t been in existence long enough for longitudinal studies (if in fact anyone is doing such).
Here in Seattle overall spending for school districts continues to go up so politicians can bloviate about money spent per student despite the fact that our results for student progress are similar to Idaho. What the districts don’t discuss is the explosive growth of administration costs, not on more teachers or classroom materials. There is something rotten in Denmark, and it’s happened all across the country.
But note that Blaine is 40% Hispanic
https://www.blaineschools.org/domain/129
I’m actually surprised that there isn’t a negative correlation, although I recognize that this wouldn’t prove causation.
Definitely there would be if the data weren’t corrected for primarily minority schools which of course would distort the findings. As long as the data is loaded in Locke’s soft wear I’d like to see some more playing around with it. Use some dummy variables for charter and minorities.
I may do a sequel post where I go through the things that do matter in more detail.
I do have statewide average outcomes by ethnicity, it’s just that there’s too many missing data points due to privacy elisions to do a correlation with other than white %age. Idaho’s mix is somewhat different than most: Largest minority is Hispanic, followed by Native American and black and Asian smaller yet. Unlike other states, the Asian skew is lower, probably since many of those identifying that way are refugees, rather than the typical high-tech migrants.
@ctlaw Blaine is the Idaho instance of larger rural West phenomenon – blue state exclaves that are turning into two class societies. I’ve got a bunch of data on that as well, but that’s definitely a new post.
I wrote about this very thing in 2011, here on Ricochet. It was in regards to the PISA (international student testing) and it shows that money has little to do with test scores.
Which means something other than money is needed to improve.
Take a look.
I agree that the relationship between spending and educational outcomes is tenuous at best. I have a couple of questions.
[1] Is the spending inflation adjusted?
[2] Is there any information as to where the money is spent? If for example, the additional funds are going to paying teachers pensions it may show up in the educational budget.
[3] At $18,000 per student and 23 students per class (https://nces.ed.gov/surveys/sass/tables/sass0708_2009324_t1s_08.asp) that is over $400K per classroom. It is clear that the money is not making it to the classroom.
Exactly. And we all know how important it is to feel good about ourselves.
It’s not so much that it feels good to spend money, as that it feels good to have bragging rights about how your child is in this or that “program” that shows you are superior to the yahoos around you, and also that you have “programs” for the unfortunate people in your district whose kids don’t have such morally and intellectually superior parents such as yourself. It’s all about “programs,” not education.
Back during the reign of George Bush the Elder, I got pre-publication papers from a couple of researchers in the DOE who were studying the school size issue. I think their papers were soon published, but what I still have somewhere in my filing cabinet (I hope) is the pre-publication versions. Anyhow, the effect of school size was that smaller schools produced better results. I don’t recall which variables the researchers tried to control for, but I’m pretty sure variables related to socio-economic status were among them. I’m pretty sure there has been more research on the topic since then that has analyzed these effects in greater detail.
I was able to use these results to argue that our district should not consolidate all of the elementary schools into one bigger one. I joined a local group that had a variety of reasons for not wanting to close the local schools, but we included these research results in our campaign literature, largely because I was the person who wrote the campaign fliers, with some editing help from others in our group. We made a big enough fuss to defeat the consolidation proposal, with the help of newspaper, radio and TV coverage. Soon after, there was another proposal which passed, and instead of closing and tearing down the small local school my youngest son had attended, they built a new one.
Unfortunately, as years went by and the children who attended the local schools grew up, the district took advantage of a loss of intense parental interest in the topic and closed the small local schools anyway. The one my son attended seems to be used now by a home schooling group. (It’s on the final leg of all three of my regular bicycle loops, and is close to the place where I got bit by a dog in March.) I am no longer involved in school district politics, and haven’t kept up.
In the last couple years when I still attended school board meetings, there was a board member who would go to Lansing to hobnob and learn which way the winds were blowing, and report back on school issues at each meeting. (This was after John Engler took much funding and control away from local districts and gave it to the state.) This board member was listened to with rapt attention. However, I noticed that there was no board member who took the initiative to go around to parents and other community members and ask them what they thought about local education issues, and then report back at each school board meeting.
By the way, I spent my freshman year of high school in a small school on the edge of a reservation that was about 1/3 Native American. It was one of the great experiences in my life. Here is a photo of our basketball team which beat out about 30 other schools, almost all of them larger than ours, to go to the state tournament. When looking for this photo I thought I might have been on it, as a student manager. (I had never so much as seen a basketball game before that year, in person or on TV. But I was the only boy who didn’t play, so they had me be student manager. The shortest of the boys on that team was listed as 5’6″ and I was shorter than that, besides not having the coordination to play. I could shoot from outside if you gave me lots of time to set up my shot, though, and it was great fun to shoot around at practices.)
I can tell you which of the guys were good students, which one lived in a tarpaper shack with no running water, which one became a tribal elder (but in looking at the web site, I see he is no longer listed) and which one was a hot shot who set a lot of scoring records that one can find on the internet.
After that year, the school closed. There were about 30 students in grades 9-12, which was a bit on the small side even for Nebraska. Nowadays the Indian kids go to a school on the reservation.
I don’t under stand Comments won’t work . Please explain
This is a commonsense assumption that falls apart in the Mountain West, in my experience. State regulations often require individual K-12 classes and rarely grant waivers for multiple grade classes. In Colorado, small rural districts in places employ 20 teachers in a district with 100 students. None of this is evidenced-based of course. Does the almost-Emile student-teacher ratio lead to excellent student performance. Of course not.
Because they are in general very poor, astonishingly ill-behaved students who have acquired very little content knowledge at the point of testing.
Now, why they are very poor, astonishingly ill-behaved students will little content knowledge …
Here’s a classic case study that is required reading for anyone interested in the school spending versus performance question.
The executive summary:
As Herrnstein and Murray learned over 20 years ago, there are certain correlates that one simply cannot discuss or even mention. Solutions, to the extent they exist, will continue to elude us if we refuse to acknowledge facts because they are inconvenient. I think many in the education community know what the real issues are but they’re either afraid to say or refuse to say.