Education Spending and Results: An Idaho Story

 

So does spending more per student improve outcomes in Idaho? In a word, no.

I have a graduate-level background in statistics. I was asked by a local conservative think tank to see whether increased spending on secondary education on Idaho has a meaningful effect on student outcomes. Idaho periodically administers its own ISAT achievement tests to elementary and secondary students. The 10th grade science test from 2014 was the latest set of results that I could obtain with a broad sampling of students with the longest exposure to public schooling. Public schools in Idaho also submit their annual budgetary information on a standardized set of accounts, 2013 was the latest available. I extracted the operating portion of districts’ budgets (educational and overhead expenses) and scaled that by the average daily attendance (ADA) in each district, showing how much was being spent per student.

Using a statistical regression test, there is no significant positive effect of increasing spending on testing outcome, across a 3x range of expenditures. The cash might as well be burnt for all the good it’s doing, based on objective testing.

Some details: The ISAT yields one of four results: Below Basic (fail), Basic, Proficient, and Advanced. The chart above shows the total of all results better than fail, which should be a minimum expectation of performance by the educational system. However, I repeated the test using both the Proficient+Advanced total, and Advanced alone, and there were no positive effects on these higher level outcomes either.

I used a total of 99 school districts’ data. While Idaho has a strong charter school movement, that data was omitted to avoid an ‘apples and oranges’ comparison because they did not administer the same test. I also omitted a number of very small districts where overhead per student might negatively bias the outcome, and a few districts that are entirely on Native American reservations. I used only operating spending, dropping categories such as capital expenditures, bond repayment or food programs that are inconsistent across districts or have no immediate effect on students.

The highest spending district in Idaho was Blaine County, home of the blue state exclaves Sun Valley, Hailey, and Ketchum. Blaine spent $18,333 in operating expenses per student. The lowest spender was rural agricultural Jefferson County, at $5,189 per student. Jefferson’s pass rate on the 10th grade test was 85.4%. Spending 3.5 times as much, Blaine had a 87.5% pass rate in its affluent district. The most effective district in Idaho, in terms of pass rate per dollar spent, was the small agricultural community of Fruitland, which has an over 20% Hispanic population.

There may be ways to improve educational outcomes, but at least in Idaho, throwing money at the problem is not one of them.

There are 36 comments.

Become a member to join the conversation. Or sign in if you're already a member.
  1. Henry Castaigne Member
    Henry Castaigne
    @HenryCastaigne

    How to advance knowledge is to post this in the Sunday newspaper and then follow it up with studies and essays and pieces every Sunday. Statistics are really hard to absorb all at once.

    People need this kind of analysis to get smarter and they should have it. Usually the papers don’t provide it.

    On a more pressing note, I’m dying to know what that $13,000 %30 dot means.

    • #1
  2. Locke On Member
    Locke On
    @LockeOn

    Henry Castaigne (View Comment):

    On a more pressing note, I’m dying to know what that $13,000 %30 dot means.

    DEpressing is the word you’re looking for, I’m afraid.  That’s a district called Plummer-Worley, that is 70% minority, most of it Native American.  And yeah, those are the actual results, not an error.  You can take a guess on what the totally on-the-rez districts would look like, and you’d be right.

     

    • #2
  3. Henry Castaigne Member
    Henry Castaigne
    @HenryCastaigne

    Locke On (View Comment):

    Henry Castaigne (View Comment)

    DEpressing is the word you’re looking for, I’m afraid. That’s a district called Plummer-Worley, that is 70% minority, most of it Native American. And yeah, those are the actual results, not an error. You can take a guess on what the totally one-the-rez districts would look like, and you’d be right.

    Could you get in touch with the editor of the Boise Weekly and talk about the difficulty of Native American education and the failure of public education?

    • #3
  4. Arizona Patriot Member
    Arizona Patriot
    @ArizonaPatriot

    I don’t need details, but was it a multivariate analysis?  I would think that a simple regression of score vs. spending would not be a proper analysis, as omitted variables could undermine the validity of the result.  For example, if the high spending is concentrated in areas with bad demographics (from a testing standpoint), omitting the demographic variable(s) would give an inaccurate result.

    • #4
  5. Locke On Member
    Locke On
    @LockeOn

    Henry Castaigne (View Comment):
    Could you get in touch with the editor of the Boise Weekly and talk about the difficulty of Native American education and the failure of public education?

    The Boise Weekly is leftist rag and unlikely to want to engage with this.  At any rate, I’m a number cruncher, not an educator.  I can point out the problem, and that the unions’ & legislators’ throw-money-at-it approach has failed, but I’m short on solutions.

    • #5
  6. Henry Castaigne Member
    Henry Castaigne
    @HenryCastaigne

    Locke On (Vie:

    The Boise Weekly is leftist rag and unlikely to want to engage with this.

    Maybe they will maybe they won’t. I ask that you call them up and talk about it. You never know unless you try.

    To a more universal point, why do Native Americans test so low?

     

     

     

    • #6
  7. Bill Nelson Member
    Bill Nelson
    @BillNelson

    The #1 indicator of education achievement success in primary and secondary schools: parents who pay attention.

     

    • #7
  8. Locke On Member
    Locke On
    @LockeOn

    Arizona Patriot (View Comment):
    I don’t need details, but was it a multivariate analysis? I would think that a simple regression of score vs. spending would not be a proper analysis, as omitted variables could undermine the validity of the result. For example, if the high spending is concentrated in areas with bad demographics (from a testing standpoint), omitting the demographic variable(s) would give an inaccurate result.

    Excellent issue and question.  I did present this as a univariate analysis for the sake of simplicity, but that was after doing a full up analysis, and I believe I have compensated for the omitted variables.  Geek_level++:

    An obvious issue is district size – small rural districts have much higher overhead per student, a subject of continual fighting between communities wanting control vs. state funders wanting efficiency.  I did a plot of spending/ADA vs. district size, there was an obvious knee in the curve, and I omitted districts below that level.

    Ethnic mix.  This is highly significant.  I can only get white %age, due to privacy issues in the dataset, but that is positively correlated with results.  However, spending level (at least in Idaho) is not highly related, due to a patchwork of subsidies on one hand, and rich/poor areas on the other.

    Poverty level.  Also highly significant.  This can be deduced from the dataset since it gives the number of students on food assistance.  One of the reasons I dropped expenditures on those programs.

    Local funding sources.  Marginally significant.  Turns out the willingness of the district’s citizens to vote supplemental levies and/or provide other purely local sources of funds is positively related to outcomes.  Logically, this shows community support for the value of education, since it does NOT matter how it’s spent.

    Once you’ve accounted for those, you are out of significant variables in the dataset, IOW funding level is not significant in either the univariate or full analysis.

    BTW, those three significant independents account for only about 50% of variance (and a scree plot descends into noise just about as quickly).  There’s a lot going on that’s not captured even in my larger dataset…

    • #8
  9. Robert McReynolds Member
    Robert McReynolds
    @RobertMcReynolds

    In Maryland’s Baltimore County the average spent per student is in the neighborhood of $18k and yet those schools are among the worst in the country.

    • #9
  10. Arizona Patriot Member
    Arizona Patriot
    @ArizonaPatriot

    Locke On (View Comment):

    Arizona Patriot (View Comment):
    I don’t need details, but was it a multivariate analysis? I would think that a simple regression of score vs. spending would not be a proper analysis, as omitted variables could undermine the validity of the result. For example, if the high spending is concentrated in areas with bad demographics (from a testing standpoint), omitting the demographic variable(s) would give an inaccurate result.

    Excellent issue and question. I did present this as a univariate analysis for the sake of simplicity, but that was after doing a full up analysis, and I believe I have compensated for the omitted variables. Geek_level++:

    An obvious issue is district size – small rural districts have much higher overhead per student, a subject of continual fighting between communities wanting control vs. state funders wanting efficiency. I did a plot of spending/ADA vs. district size, there was an obvious knee in the curve, and I omitted districts below that level.

    Ethnic mix. This is highly significant. I can only get white %age, due to privacy issues in the dataset, but that is positively correlated with results. However, spending level (at least in Idaho) is not highly related, due to a patchwork of subsidies on one hand, and rich/poor areas on the other.

    Poverty level. Also highly significant. This can be deduced from the dataset since it gives the number of students on food assistance. One of the reasons I dropped expenditures on those programs.

    Local funding sources. Marginally significant. Turns out the willingness of the district’s citizens to vote supplemental levies and/or provide other purely local sources of funds is positively related to outcomes. Logically, this shows community support for the value of education, since it does NOT matter how it’s spent.

    Once you’ve accounted for those, you are out of significant variables in the dataset, IOW funding level is not significant in either the univariate or full analysis.

    BTW, those three significant independents account for only about 50% of variance (and a scree plot descends into noise just about as quickly). There’s a lot going on that’s not captured even in my larger dataset…

    Based on Charles Murray’s work in The Bell Curve, I suspect that IQ and parental socioeconomic status (SES) would help explain more of the variance.  I realize that you probably don’t have such information in your dataset.  Poverty level would be something of a proxy for SES, and %white would probably be something of a proxy for IQ.

    I don’t know Idaho well enough to comment on urban/rural differences.  I would generally expect urban areas to have higher costs, due to higher land values and cost of living (and thus the need to pay teachers more).  This might partially offset the district size issue.

    • #10
  11. Max Ledoux Admin
    Max Ledoux
    @Max

    I’m promoting this post to the Main Feed for 4PM ET. Comments won’t work until then.

    • #11
  12. danok1 Member
    danok1
    @danok1

    @lockeon:

    I’d guess that most of the Educrats would say that the ISAT doesn’t really measure student learning or “critical thinking” (or whatever they want to claim as the “right” measure for student outcomes) so your pretty little analysis means nothing…give us more money!!!

    So, do you have any stats on how the ISAT correlates with later student achievement (e.g., admission to “top” universities)?

    • #12
  13. Mike H Coolidge
    Mike H
    @MikeH

    Unfortunately, objectivity doesn’t matter. It feels good to spend more money on education so the majority of people are going to support it.

    • #13
  14. Locke On Member
    Locke On
    @LockeOn

    danok1 (View Comment):
    @lockeon:

    So, do you have any stats on how the ISAT correlates with later student achievement (e.g., admission to “top” universities)?

    No, the tests haven’t been in existence long enough for longitudinal studies (if in fact anyone is doing such).

     

    • #14
  15. Goldwaterwoman Thatcher
    Goldwaterwoman
    @goldwaterwoman

    Locke On: There may be ways to improve educational outcomes, but at least in Idaho, throwing money at the problem is not one of them.

    Here in Seattle overall spending for school districts continues to go up so politicians can bloviate about money spent per student despite the fact that our results for student progress are similar to Idaho.  What the districts don’t discuss  is the explosive growth of administration costs, not on more teachers or classroom materials. There is something rotten in Denmark, and it’s happened all across the country.

    • #15
  16. ctlaw Coolidge
    ctlaw
    @ctlaw

    Locke On: The highest spending district in Idaho was Blaine County, home of the blue state exclaves Sun Valley, Hailey, and Ketchum. Blaine spent $18,333 in operating expenses per student. The lowest spender was rural agricultural Jefferson County, at $5,189 per student. Jefferson’s pass rate on the 10th grade test was 85.4%. Spending 3.5 times as much, Blaine had a 87.5% pass rate in its affluent district. The most effective district in Idaho, in terms of pass rate per dollar spent, was the small agricultural community of Fruitland, which has an over 20% Hispanic population.

    But note that Blaine is 40% Hispanic

    https://www.blaineschools.org/domain/129

     

    • #16
  17. Larry3435 Member
    Larry3435
    @Larry3435

    I’m actually surprised that there isn’t a negative correlation, although I recognize that this wouldn’t prove causation.

    • #17
  18. I Walton Member
    I Walton
    @IWalton

    Larry3435 (View Comment):
    I’m actually surprised that there isn’t a negative correlation, although I recognize that this wouldn’t prove causation.

    Definitely there would be if the data weren’t corrected for primarily minority schools which of course would distort the findings.  As long as the data is loaded in Locke’s  soft wear  I’d like to see some more playing around with it. Use some dummy variables for charter and minorities.

    • #18
  19. Locke On Member
    Locke On
    @LockeOn

    I may do a sequel post where I go through the things that do matter in more detail.

    I do have statewide average outcomes by ethnicity, it’s just that there’s too many missing data points due to privacy elisions to do a correlation with other than white %age.  Idaho’s mix is somewhat different than most: Largest minority is Hispanic, followed by Native American and black and Asian smaller yet.  Unlike other states, the Asian skew is lower, probably since many of those identifying that way are refugees, rather than the typical high-tech migrants.

    @ctlaw Blaine is the Idaho instance of larger rural West phenomenon – blue state exclaves that are turning into two class societies.  I’ve got a bunch of data on that as well, but that’s definitely a new post.

     

    • #19
  20. Instugator Thatcher
    Instugator
    @Instugator

    Locke On (View Comment):
    I may do a sequel post where I go through the things that do matter in more detail.

    I wrote about this very thing in 2011, here on Ricochet. It was in regards to the PISA (international student testing) and it shows that money has little to do with test scores.

    Which means something other than money is needed to improve.

    Take a look.

    • #20
  21. Frank Monaldo Member
    Frank Monaldo
    @FrankMonaldo

    I agree that the relationship between spending and educational outcomes is tenuous at best.  I have a couple of questions.

    [1] Is the spending inflation adjusted?

    [2] Is there any information as to where the money is spent? If for example, the additional funds are going to paying teachers pensions it may show up in the educational budget.

    [3] At $18,000 per student and 23 students per class (https://nces.ed.gov/surveys/sass/tables/sass0708_2009324_t1s_08.asp) that is over $400K per classroom. It is clear that the money is not making it to the classroom.

     

    • #21
  22. Songwriter Member
    Songwriter
    @user_19450

    Mike H (View Comment):
    Unfortunately, objectivity doesn’t matter. It feels good to spend more money on education so the majority of people are going to support it.

    Exactly. And we all know how important it is to feel good about ourselves.

    • #22
  23. Locke On Member
    Locke On
    @LockeOn

    Frank Monaldo (View Comment):
    I agree that the relationship between spending and educational outcomes is tenuous at best. I have a couple of questions.

    [1] Is the spending inflation adjusted?

    [2] Is there any information as to where the money is spent? If for example, the additional funds are going to paying teachers pensions it may show up in the educational budget.

    [3] At $18,000 per student and 23 students per class (https://nces.ed.gov/surveys/sass/tables/sass0708_2009324_t1s_08.asp) that is over $400K per classroom. It is clear that the money is not making it to the classroom.

    1.  No, and it makes no difference since I’m using a single year’s budget numbers.  I’m making what I think is a reasonable assumption that such a snapshot is reflective of longer term spending in a district.
    2. Yes, it is.  As stated in the OP, I used the rollup lines for instructional and overhead spending.  Current account spending on pensions would therefore show up.  I did not do a district-by-district check on whether pensions were funded on a prepay or pay-as-you-go basis.  There is a statewide teacher retirement fund, that is in pretty good shape on an actuarial basis, so I’d imagine that prepay is the rule.
    3. “not making it to the classroom” – you can’t conclude that from this study.  They might very well be spending the wad on higher teacher salaries and classroom gizmos.  But it’s being spent ineffectively as far as student outcomes.  ETA:  They could also be spending the money on more teachers/smaller classes, which would invalidate the assumption in your comment.
    • #23
  24. The Reticulator Member
    The Reticulator
    @TheReticulator

    Mike H (View Comment):
    Unfortunately, objectivity doesn’t matter. It feels good to spend more money on education so the majority of people are going to support it.

    It’s not so much that it feels good to spend money, as that it feels good to have bragging rights about how your child is in this or that “program” that shows you are superior to the yahoos around you, and also that you have “programs” for the unfortunate people in your district whose kids don’t have such morally and intellectually superior parents such as yourself.   It’s all about “programs,” not education.

    • #24
  25. The Reticulator Member
    The Reticulator
    @TheReticulator

    Locke On: I also omitted a number of very small districts where overhead per student might negatively bias the outcome, and a few districts that are entirely on Native American reservations.

    Back during the reign of George Bush the Elder, I got pre-publication papers from a couple of researchers in the DOE who were studying the school size issue. I think their papers were soon published, but what I still have somewhere in my filing cabinet (I hope) is the pre-publication versions. Anyhow, the effect of school size was that smaller schools produced better results. I don’t recall which variables the researchers tried to control for, but I’m pretty sure variables related to socio-economic status were among them.  I’m pretty sure there has been more research on the topic since then that has analyzed these effects in greater detail.

    I was able to use these results to argue that our district should not consolidate all of the elementary schools into one bigger one.  I joined a local group that had a variety of reasons for not wanting to close the local schools, but we included these research results in our campaign literature, largely because I was the person who wrote the campaign fliers, with some editing help from others in our group.  We made a big enough fuss to defeat the consolidation proposal, with the help of newspaper, radio and TV coverage.  Soon after, there was another proposal which passed, and instead of closing and tearing down the small local school my youngest son had attended, they built a new one.

    Unfortunately, as years went by and the children who attended the local schools grew up, the district took advantage of a loss of intense parental interest in the topic and closed the small local schools anyway.  The one my son attended seems to be used now by a home schooling group. (It’s on the final leg of all three of my regular bicycle loops, and is close to the place where I got bit by a dog in March.)  I am no longer involved in school district politics, and haven’t kept up.

    In the last couple years when I still attended school board meetings, there was a board member who would go to Lansing to hobnob and learn which way the winds were blowing, and report back on school issues at each meeting. (This was after John Engler took much funding and control away from local districts and gave it to the state.) This board member was listened to with rapt attention.  However, I noticed that there was no board member who took the initiative to go around to parents and other community members and ask them what they thought about local education issues, and then report back at each school board meeting.

    By the way, I spent my freshman year of high school in a small school on the edge of a reservation that was about 1/3 Native American. It was one of the great experiences in my life. Here is a photo of our basketball team which beat out about 30 other schools, almost all of them larger than ours, to go to the state tournament.  When looking for this photo I thought I might have been on it, as a student manager. (I had never so much as seen a basketball game before that year, in person or on TV. But I was the only boy who didn’t play, so they had me be student manager. The shortest of the boys on that team was listed as 5’6″ and I was shorter than that, besides not having the coordination to play. I could shoot from outside if you gave me lots of time to set up my shot, though, and it was great fun to shoot around at practices.)

    I can tell you which of the guys were good students, which one lived in a tarpaper shack with no running water, which one became a tribal elder (but in looking at the web site, I see he is no longer listed) and which one was a hot shot who set a lot of scoring records that one can find on the internet.

    After that year, the school closed. There were about 30 students in grades 9-12, which was a bit on the small side even for Nebraska. Nowadays the Indian kids go to a school on the reservation.

    • #25
  26. Rocket Surgeon Member
    Rocket Surgeon
    @RocketSurgeon

    Max Ledoux (View Comment):
    I’m promoting this post to the Main Feed for 4PM ET. Comments won’t work until then.

    I don’t under stand Comments won’t work .  Please explain

    • #26
  27. Quake Voter Member
    Quake Voter
    @QuakeVoter

    Arizona Patriot (View Comment):
    I would generally expect urban areas to have higher costs, due to higher land values and cost of living (and thus the need to pay teachers more). This might partially offset the district size issue.

    This is a commonsense assumption that falls apart in the Mountain West, in my experience.  State regulations often require individual K-12 classes and rarely grant waivers for multiple grade classes.  In Colorado, small rural districts in places employ 20 teachers in a district with 100 students.  None of this is evidenced-based of course.  Does the almost-Emile student-teacher ratio lead to excellent student performance.  Of course not.

    • #27
  28. Quake Voter Member
    Quake Voter
    @QuakeVoter

    Henry Castaigne (View Comment):
    To a more universal point, why do Native Americans test so low?

    Because they are in general very poor, astonishingly ill-behaved students who have acquired very little content knowledge at the point of testing.

    Now, why they are very poor, astonishingly ill-behaved students will little content knowledge …

    • #28
  29. Johnny Dubya Member
    Johnny Dubya
    @JohnnyDubya

    Here’s a classic case study that is required reading for anyone interested in the school spending versus performance question.

    The executive summary:

    For decades critics of the public schools have been saying, “You can’t solve educational problems by throwing money at them.” The education establishment and its supporters have replied, “No one’s ever tried.” In Kansas City they did try. To improve the education of black students and encourage desegregation, a federal judge invited the Kansas City, Missouri, School District to come up with a cost-is-no-object educational plan and ordered local and state taxpayers to find the money to pay for it.

    Kansas City spent as much as $11,700 per pupil–more money per pupil, on a cost of living adjusted basis, than any other of the 280 largest districts in the country. The money bought higher teachers’ salaries, 15 new schools, and such amenities as an Olympic-sized swimming pool with an underwater viewing room, television and animation studios, a robotics lab, a 25-acre wildlife sanctuary, a zoo, a model United Nations with simultaneous translation capability, and field trips to Mexico and Senegal. The student-teacher ratio was 12 or 13 to 1, the lowest of any major school district in the country.

    The results were dismal. Test scores did not rise; the black-white gap did not diminish; and there was less, not greater, integration.

    The Kansas City experiment suggests that, indeed, educational problems can’t be solved by throwing money at them, that the structural problems of our current educational system are far more important than a lack of material resources, and that the focus on desegregation diverted attention from the real problem, low achievement.

     

    • #29
  30. drlorentz Member
    drlorentz
    @drlorentz

    Arizona Patriot (View Comment):
    Based on Charles Murray’s work in The Bell Curve, I suspect that IQ and parental socioeconomic status (SES) would help explain more of the variance.

    As Herrnstein and Murray learned over 20 years ago, there are certain correlates that one simply cannot discuss or even mention. Solutions, to the extent they exist, will continue to elude us if we refuse to acknowledge facts because they are inconvenient. I think many in the education community know what the real issues are but they’re either afraid to say or refuse to say.

    • #30

Comments are closed because this post is more than six months old. Please write a new post if you would like to continue this conversation.