Ricochet is the best place on the internet to discuss the issues of the day, either through commenting on posts or writing your own for our active and dynamic community in a fully moderated environment. In addition, the Ricochet Audio Network offers over 50 original podcasts with new episodes released every day.
Good Science is Hard to Do
Megan McArdle’s evisceration of the idea that government subsidies for contraception have been proven to reduce abortion rates is something every conservative should study and take notes on. I commend the whole piece, especially her conclusion on one of the most-cited studies in favor of the more-contraceptives-less-abortions argument:
[The researchers] tell you that Colorado gave out LARCs [long-acting reversible contraceptives]; they tell you that birthrates and abortions fell. They don’t dwell on the simultaneous fall in these areas at the national level, which is somewhat mysterious but may be fallout from the financial crisis. They have no way to establish causality except for an inadequate control group that doesn’t even show a substantial difference in teen abortion rates, a fact that they appear to have forgotten in other sections of the paper. (They also mention that the overall decline in Colorado’s birthrates was concentrated among low-income women in the studied counties, but that’s not actually very interesting, because early motherhood and unintended pregnancy are also concentrated among low-income women.)
The authors basically concede that they cannot come close to establishing causality, because the summary conclusion is a weasel: LARCs may contribute to a decline in fertility and abortion among high-risk women. How much? We don’t know, but look at the pretty graphs!
A lack for appreciation for just how difficult it is to produce good research is among the most dangerous aspects of the study-loving, “Science!” boosters on the left. Conducting a rigorous, well-researched, falsifiable experiment that holds up to scrutiny and offers anything approaching a meaningful addition to human knowledge is devilishly difficult under the best of circumstances. It’s massively more complicated when one’s subjects are thinking, adaptable, stubborn, difficult-to-control human beings.
Conservatives should — indeed, as truth-seekers, must — put a high premium on scientific method and follow the evidence wherever it leads. But we should always remind ourselves and others not to jump to conclusions, or to mistake the (honest, well-intended) trappings of empiricism for the real thing.
Published in General
Great point. Richard Feynman called this ‘Cargo Cult Science’ – people who think they are being ‘scientific’ by exercising the superficial trappings of science without the discipline, attention to detail, thoughness and humility that is required to actually do good science.
He had scathing things to say about the state of the ‘social’ sciences, economics, etc. He died before the big global warming debate erupted – I’d be curious to know what he’d think of the state of that ‘science’. He was a huge opponent of the notion that the consensus opinion of scientists was some kind of trump card.
I think it would be interesting to set up an experiment where you offer to pay people to accept LARCs or pay them more to get more-or-less permanent contraception and see what happens to birth/abortion rates.
Perhaps we offer first-time, single mothers coming in to a hospital while on Medicaid the sweet deal of a $10,000 payment to accept tubal ligation in order to prevent all future pregnancies? Maybe we set up a program where we offer 18-yr-old males $15,000 to get a vasectomy and slide that number down with each consecutive year?
Any of these would have a hugely salutory effect on the birth/abortion rate at a minimum amount of cost.
These things have unforeseen consequences. Better to stop trying to thumb the scale with government intervention.
My, admittedly, unscientific reaction to this is to file as “doubtful” all “studies” and poll’s. I know those are two really different things, however, my newsfeed is constantly bombarded with studies and polls that tell me what I should think.
As a layman (non-scientist, non-statistician) who consumes news reports as a way to inform myself, the endless stream of new scientific insight produced becomes white noise or, more likely, propaganda for a certain point-of-view (right or left).
In short, I don’t believe them. Why should I?
Having worked in science most of my professional life, I wholeheartedly agree with almost everything in McArdle’s article.
But
I have also noticed over the years that most of the errors and fallacies she complains about are committed equally by both sides of the political spectrum (even while both of those sides accuse the other of being “anti-science”). Indeed, many professional scientists – even professors with PhDs and hundreds of publications – constantly embrace unwarranted conclusions when it comes to scientific “studies” outside of their field of expertise.
A great example is the Oregon Medicaid study which came out a few years ago. It was orders of magnitude more scientifically sound than this Colorado contraception article appears to be, yet it was still incredibly limited in its ability to make widespread predictions about the effects of government-subsidized healthcare. Yet many (serious) conservatives immediately latched onto its weak effects as rock-solid proof that government-run healthcare can never work.
It is absolutely true that science is hard – much harder than most people want to admit (especially those who are strongly “pro-science”). There is very little which one study can say with any degree of certainty.
But we need to be honest with ourselves: humans are very bad at handling uncertainty. We – both individually and collectively – have an unstoppable need for “objective” findings to inform our decisions. And when that objective information simply doesn’t exist, we will not be able to resist the temptation to take poor science and pretend it is objective.
While I applaud McArdle for fighting the good fight, this is a problem which will never go away, and likely never get much better, either.
I can see why the Colorado study is so appealing, though. It seems reasonable—indeed, intuitively obvious— that if you give young women who don’t want to have children right now a long-acting contraceptive that they don’t have to remember to use, the number of unintended pregnancies would drop, and with these, abortions as well as out-of-wedlock births.
The issue that I have with this is that we are already neck-deep in subsidizing people for having children out of wedlock to the tune of many, many times that $10,000. A single birth in the Medicaid system will likely exceed that cost.
A $10,000 financial incentive sounds like a low cost insurance policy which prevents each and every subsequent pregnancy that will have the attendant costs of Medicaid, Education, WIC, Section 8, SCHIP, School Lunch and every other income redistribution and welfare program under the sun associated with it.
Thank you, Mendel!
In my world, Christian clergy persons who disdain science when it comes to, say, evolution will fall over themselves in their eagerness to cite “scientific studies” that prove the efficacy of prayer.
On the other hand, we have to act (or not act) on the basis of imperfect data. The government is going to do—or not do—something. I’d feel better about government programs if it was understood that each was a kind of experiment in itself, and thus to be subjected to rigorous and frequent review to see whether the effects were the ones desired or expected.
I like this line from McArdle’s piece: Instead of forcing a hard moral choice between the autonomy of the woman and the value of the potential life that is terminated, belief in the birth control fairy lets us off the hook.
In a recent podcast, Milt Rosenberg (noted social psychologist), referred to social scientoids rather than social scientists. I can’t recall his words exactly, but the gist was that social ‘science’ is not.
You guys are asking too much if you want the scientific method to be applied successfully in this area. It’s cargo cult all the way down.
We do have an experiment to see if a radical increase in contraception availability will decrease unwanted pregnancies (and abortion): it’s called the last fifty years.
What happened is that sexual and parenting mores changed more than the Pill and condoms reduced pregnancies, such that out of wedlock births (and STDs) are higher, not lower, than they were back in the ’50s.
I’ll also point out that women have babies in bad situations for any number of reasons, and lack of access to contraceptives is rarely among them. (IIRC, Guttmacher once put out a study showing that only about 10% of women having abortions didn’t have access to contraceptives.) They want someone to love. They want to keep the dad around. They get more money from the government. They don’t see any other path in life. Whatever it is, this “study” is only going to attract women who are aggressive about not getting pregnant.
I think he described them as “the social disciplines,” which also struck me as an interesting idea.
I meant to post this earlier:
Yes, but with such poor controls. :(
Further on Mendel’s point above, science is not only hard, but it infrequently produces interesting results. If you hire a huge number of people to practice science, you wind up in a position where the incentive to produce seemingly interesting results becomes immense, which results in “science” that produces this result:
Or this:
It suggests that we need less money in science, not more.
Therefore, there should be a heavy bias towards not acting at all. Society is an ecosystem, and in nature we’ve learned that if we don’t really understand the ecosystem (and we never do), it’s probably a good idea to just leave it alone rather than try to intervene to ‘improve’ it, since it’s managed to get along just fine without us and our interventions might just have unintended consequences.
And if we must intervene, it’s much better do to it at the smallest possible level. Not only do the consequences of mistakes have less scope, but when you have imperfect information the best thing you can do is to try many different things and learn which ones work and be ready to abandon the ones that don’t.
And of course, the ultimate in local government is the individual. So we should be biased strongly towards leaving people alone, and if there must be intervention to fix a problem it should be local and closely targeted and set up in a way that allows it to be abandoned if it doesn’t meet the results we set for it.
This is the key. Not just government programs, but all sorts of “interventions” (I’m thinking protest movements, for example) don’t have an off switch.
Nate Silver (of FiveThirtyEight) wrote a book called The Signal and the Noise that deals with this theme very well. He’s a strong proponent of Bayesian statistics because of the way it handles uncertainty and subjectivity in an upfront manner.
It’s a great book! I’ve been planning to write a book review of it, but haven’t gotten around to it.
Anyone notice Silver has fallen out of favor with the left ever since some of his work didn’t validate what they believe? He’s still a lefty, but an honest one. He goes mostly where the data takes him. Being a professional gambler forces you to look at the data impartially, lest you go bankrupt.
Tom,
Yes those Humans do and say the darndest things. Why can’t they just be physical objects that bounce around according to natural laws so we could calculate their behavior better?
Why indeed.
Regards,
Jim
Please do!
One thing I appreciate about Nate Silver, and think helps keep him honest, is that politics is not his first or only interest (that would be baseball). He moved from the NYT to ESPN, after all.
A savvy (R) candidate might even be able to get some (grudging) positive press from FiveThiryEight if she or he came out in favour of legalizing prediction markets…
I am related to some pretty prominent – and quite highly respected – physicists.
Some years ago, I realized that when it comes to anything outside their core specialties, they don’t have any logical rigor. See dietary fat floating in the bloodstream? Obviously it hardens the arteries. “An Inconvenient Truth“? A great scientific movie. The farther the scientific endeavor is from their core competency, the more they believe whatever is being shoveled, with no questions asked.
If the New York Times reports it as scientifically valid, then all correlated things are – of course! – causally linked.
I was just writing something about this on another thread: Holy cow, do I know a lot of left-liberals. That’s who I’m preaching to, bless them. And of course, I have to assume that minds can change, or there wouldn’t be much point in doing what I do. And listen: These are very nice, bright people—not the idiotic, drooling fascists of Ricochetti fantasy. On the other hand, they’re wrong about a lot of stuff. And are too often not interested in considering alternative viewpoints.
Well, few are. Few of us make all of our decisions by carefully gathering and examining evidence, though most of us will do this in one area of life. Hence, the more you know about a subject (the War in Iraq, the relationship between law enforcement and young black men, the behavior of the German churches under Naziism) the more complicated it is. The less you know, the more absolutely confident you can be that you’ve got the answers. For Noam Chomsky, linguistics is complicated… but war and peace are simple.
Like. :)
I’m going to make a wild guess here: your physicist relatives are all hard Leftists? Amiright or amiright? The trouble with them is not a lack of rigor per se; it’s putting politics ahead of all else. That disease of the Left transcends academic disciplines.
Hmm.. Chomsky’s genius (or hubris, depending on who you ask) as a linguist lay in simplifying linguistics (oversimplifying, some would say, though I’d disagree). I wonder if he sees linguistics as complicated, or simply as transparent.
War and peace, on the other hand, could simply be opaque. Not simple as in easily-understood, but just topics that moral people have a duty to be righteous about, even if they don’t understand them well.
After all, much of morality is salutary even if we don’t understand it. People who just believe stealing is wrong and promiscuity is gross and who act accordingly, even if they can’t articulate why these things are bad, may be in better shape than those who can articulate the whys of moral belief but don’t feel compelled to act according to those beliefs.
Linguistics isn’t morally burdensome. Therefore good people don’t have to pretend to understand it. War and peace are.
Immanuel Velikovsky wrote truly awful books in the 1970’s claiming that the Bible and other historical documents with supernatural phenomenon should be taken literally and we should figure out how the physics might have worked to cause the events that were described. So, he theorized that Jupiter spit out a planet (Venus), and that Venus went careening past Mars, knocking it out of its orbit. Venus then flew past earth, and carbohydrates in its ‘tail’ rained down on earth and provided manna from heaven (Velikovsky confused hydrocarbons and carbohydrates, or counted on his audience to not know the difference). Then Mars flew by and stopped the Earth’s rotation for a day, etc.
It was utter nonsense, but got a lot of traction with the public and a few nice blurbs from serious scientists. Velikovsky cleverly crossed many disciplines in his work – geology, history, anthropology, astronomy, etc. That ensured there was not one person expert enough to deny all of it.
So what happened is that a physicist would read it and say, “Oh, of course the physics are totally nutty. But I’m very impressed with his historical scholarship and his understanding of geology.” And then a geologist would say, “well, of course the geology in the book is completely idiotic, but I’m very impressed with his understanding of astronomy and history.” And a history prof would say, “Of course the history is completely wrong, but he sure knows his geology.” The experts in one field were easily baffled by nonsense in another, but being ‘experts’ felt competent to judge its quality anyway.
So very few were willing to dismiss the book outright, and that led even people in some of those fields to give the book a second look. I actually had a physics professor ask me to read his manuscript for a book he wrote defending Velikovsky’s work. All that did was make me determined to go to a different university to study physics.
This is akin to the notion that federalism allows each state to try out different policies, and let them act like “50 different laboratories”.
While I absolutely agree with both of these sentiments, from a scientific perspective it’s still pretty hard to objectively determine how well a certain government program (or lack thereof) contributed to any specific outcome.
In the case of national programs, there is no real control group at all – we can only compare the current outcome with outcomes prior to enactment, or to other countries, neither of which pass much scientific muster.
In the case of state programs, we have somewhat better control groups, but still have a ton of confounding factors. A main confounding factor is that a state with the political desire to make a certain change must a priori have a much different type of constituent than a state which does not make that change, which would be a strong enough bias to disqualify almost any clinical trial with a similar underlying factor.
The best we can hope for is that either a) certain policies work so well that we can trust our gut feeling, or b) certain policies will be so spectacularly bad that we can rule them out even in the absence of scientific rigor.
When evaluating policies, we also have to consider the time lag. A policy can work when first implemented because everyone is highly motivated to do a good job, media coverage is high, etc. The true test is how it works 10 years down the road, but by then both sides have ammunition for/against it.
Supply side tax cuts can cause a short-term deficit, but the added growth over the long term may make up for it. But by the time that growth happens there might be a different president and he will take credit for the growth and blame the deficit on his predecessor.
Some policies (many policies…) are intentionally designed so that the bad effects are pushed to the ‘out years’ so they can be blamed on someone else. That makes it even harder to assign cause and effect.
One problem with determining the value or cost of tax policy is that such policies are often instituted as the result of some other fiscal shock, and it can be hard to separate cause and effect. The economy takes a downturn, so you cut taxes. To partisans on the other side, tax cuts = a downturn. Or, the economy is booming, so the government gets more revenue and enacts new spending. Therefore, spending = a booming economy.
Economics just isn’t a science. It’s not amenable to mathematical analysis the way physics or chemistry is. There are no control groups, no do-overs. Even collecting the required data is subject to bias and other errors.
And even historical analysis is suspect because the economy is an adaptive system and always changing. A tax cut or hike applied to today’s economy could have a radically different effect than an identical tax cut or hike applied to the economy we had a decade ago. So, there’s a limit to what we can learn from the past, and no way to construct experiments that we can use to evaluate the present. The future is unpredictable because innovation and change in a complex system is a random walk.
Where does that leave us? Logic, and maybe humility. Maybe we shouldn’t be trying to change so much, since we really don’t know what we’re doing.
Let’s not kid ourselves: this same phenomenon is alive and kicking on the right as well.
I continually read/hear conservative scientists make embarrassing fallacies when opining on subjects outside of their expertise. It even happens regularly on Ricochet, even from some of the more prominent and public scientists here.
A classic example is global warming: a conservative scientist with no background in climatology will criticize a pro-global warming study and (often correctly) point out gaping holes in the conclusions and/or data of said study. However, that person then usually a) ignores the rest of the field/context of climate science, which is staggeringly huge, and b) claims that their critique somehow “proves” that global warming is not occurring, when all they proved was that a particular study was shoddy.
For a more concrete example, look at Ben Carson’s pronouncement that we know homosexuality is not an innate trait since some straight male prisoners have sex with other men while in prison. For a candidate whose main appeal is “Dammit, Jim, I’m a scientist, not a politician!” that was something of a howler.