Ricochet is the best place on the internet to discuss the issues of the day, either through commenting on posts or writing your own for our active and dynamic community in a fully moderated environment. In addition, the Ricochet Audio Network offers over 50 original podcasts with new episodes released every day.
Searching for Problems and Ignoring Solutions
I took statistics in college, and we studied it in medical school, but for some reason it didn’t really affect me. Then, later in my career, I gradually came to the realization that many of the decisions I was making for my patients were based on guesswork. Educated guesswork. But still, I was mostly guessing. That is a terrifying realization for a physician to make. Although in my view, it’s more terrifying when a physician doesn’t realize it.
Anyway, at that point, I started studying statistics in earnest. I’m good at math, and had always excelled at calculus and so on in school. But I slowly reached the conclusion that statistics isn’t really math. At least, not in my view. It’s a way of looking at the world. In fact, the world doesn’t make much sense, until you understand statistics. At that point, things become a little clearer (or, perhaps, a bit less unclear), and decision making becomes simpler. Again – the use of statistics does not make things more complex. It makes things simpler.
I’ve given hundreds of speeches to doctors’ groups all over the country (mostly about heart disease), and I spend a lot of time on statistics. The number one complaint I get on my reviews is, “I’m not a mathematician! I’m a doctor! Talk about medicine, not math!” I respond simply that medicine is complex, and it’s virtually impossible to understand if you don’t understand statistics. Math is simply the language we use to precisely describe complicated things. I teach about math, because I don’t know how else to explain how heart disease works. But after having explained my point to countless doctors, and countless patients, I’ve noticed a few trends about how people look at complex problems. Perhaps you’ll recognize these trends, based on your experiences:
I think that in general, most doctors order way too many tests. They don’t necessarily understand exactly how those tests are done, or what they measure exactly, or what to do with the results. But if they think someone has heart disease, they just start checking every box on the “Cardiovascular” panel. I explained part of my concern about this in a previous post:
A couple years ago I was giving a lecture to a group of Cardiologists about advanced cardiovascular testing, and someone asked if I used a certain lab test. I answered, no, that I found it unhelpful. He protested that it was an accurate test. I agreed, but replied, “To me, it’s like an Obama sticker on a Prius. It may be accurate, but it doesn’t add any information that you didn’t already know.”
So I spend a lot of time trying to get doctors to think backwards, sort of. What can be done if the person has this disease? What tests will help you determine your course of action? What tests are unhelpful, and therefore potentially misleading? This is a big, complicated topic well beyond the scope of this post. But my point is that lots of very intelligent physicians struggle with relatively straightforward data analysis. And they do this for a living.
Imagine those with no background in math.
I’ve noticed that a patient’s inclination to treat a disease is often inversely proportional to their inclination to collect data about that disease. And I think that perhaps that dichotomy might help explain many of our social problems today. Let me try to explain. See if you agree with my premise…
“Susan” had dinner at her club last month with a patient of mine, who happens to be a retired cardiovascular surgeon. He went on at some length about how impressed he was with my efforts to approach heart disease in a rational, evidence-based manner.
Susan is very fit and focused on her health, because everybody in her family dies young of heart disease. Her total cholesterol is always over 300 despite her best efforts. She refuses to take statins or other ‘cholesterol drugs,’ but understands that she has a problem. She was so impressed by what her friend said that she paid her money and joined my practice. Our first visit was fascinating:
Susan: “I really look forward to working with you. Dr. Smith said you’re really good at heart disease.”
Me: “Thanks! I do the best I can. How can I help you?”
Susan: “I have really high cholesterol and a lousy family history. But I don’t want to put a bunch of chemicals in my body, and all my previous physicians have tried to put me on a bunch of drugs. I’ll be interested to hear your advice.”
Me: “Sure. Do you have any of your previous labs?”
Susan: “Here’s the last set of labs from my previous doc.”
* hands me a big stack of papers *
Me: * starts paging through every lab test I’ve ever heard of, and some that I hadn’t… *
“Wow.”
Susan: “Wow? What do you mean?”
I’m not sure what to say here. Either her previous physician had no idea what she was doing, or she was trying to show off by ordering lots of exotic tests. I suppose it doesn’t matter which one is true. I’m trying not to step on toes, so I ask, “So, um, what did your previous doc say about all this?”
Susan: “She tried to put me on a statin. But I’ve read that statins are bad for you.”
Me: “I can believe that.”
Susan: “You believe that statins are bad for you?”
Me: “No. I can believe that you read that.”
Susan: “What do you mean?”
Me: “I mean that if your previous doc had read your family history, and had looked at these labs, and decided not to treat, that I would call that doctor on the phone and ask what on earth she was thinking.”
Susan: “I don’t want to put a bunch of chemicals in my body.”
Me: “Do you take any meds at all?”
Susan: “No way. Like I said, I don’t want to put a bunch of chemicals in my body.”
Me: “Do you take any supplements?”
Not surprisingly, Susan pulled a Ziploc bag out of her purse, filled with every supplement I’ve ever heard of, and some that I hadn’t. I said, “Wow – that’s a lot of chemicals that you’re putting into your body.” I grab a bottle at random and show it to her. “What does the research say about this one?”
Susan: “What do you mean?”
Me: “Do people who take it live longer? Which people? Are you one of those people? How can we tell?”
Susan: * starting to get understandably defensive *
“Look, can we just repeat these labs, and see what we find?”
Me: “Not if we’re not going to treat it. If you’ve decided not to treat your heart disease, that’s fine. That’s your prerogative. But if we’re considering treatment, I think we should check some basic stuff, look for root causes of disease, and decide on a rational course of action here.”
Susan: “What about my SDMA? What about all those other inflammation markers?”
Me: “Those may prove to be relevant someday, but right now they’re very poorly understood. Let’s work with what we know – what we have evidence to support. We already have what we need. Let’s develop a plan of attack, based on your particular situation. Find the disease. Fix the disease. Move on with our lives. Checking the same meaningless tests over and over again is unhelpful. And if that’s your plan of action, then you don’t need me.”
Well, things went downhill from there.
She wanted to do more tests, but didn’t want to treat. I wanted to treat, but didn’t want to do more tests. The only reason the conversation went on as long as it did was that her husband was there, and didn’t want to walk away from potentially life-saving treatment.
I found it fascinating that she was so unwilling to consider treatments, but she was so insistent on doing more and more testing. Why do all these tests to find a problem that you have no interest in fixing?
Consider the global warming crowd. They want more and more computer models. Is the earth warming by 1.7 degrees? Or is it 1.9 degrees? Let’s do more studies!
Well, ok. But if fossil fuels are dangerous, perhaps we might consider nuclear power, which results in only warm water and tiny amounts of nuclear waste which can be easily contained and managed. Ok. Let’s get to work.
No. Let’s do more and more computer models. Which we refuse to act on.
How many studies have been done on the problems of children of broken families, pipelines, welfare programs, ethanol fuel, virus mitigation strategies, immigration policies, and so on? We keep doing more studies. And more and more and more.
But it’s easier to study these things than to fix them. So we search for problems that we fully intend to ignore. This seems odd, but it is not unusual behavior.
Now, I understand, up to a point. Our fears of statins, nuclear power, intact families, and so on prevent us from considering rational treatments, so we do more studies. Got to spend that grant money somehow, right? And maybe we might find a more palatable solution. Maybe.
But what if we don’t?
It seems to me that sometimes the more data someone wants, the less interested they are in fixing the problem they’re studying. Or something.
One problem here is probably that in order to improve most of our social problems, our government would need to do less, and spend less, rather than more. No politician can run for office on that, so they avoid exploring solutions (which might not work, which would look bad) and they spend more money on studies.
But I don’t think that’s it, entirely. I think there may be a parallel between excessive government studies and my patients who want every test under the sun run on them, regardless of the evidence behind these tests, because they want to “do something” about a disease they have no interest in treating.
“Susan’s” approach to heart disease seems similar to Nancy Pelosi’s approach to homelessness. And is similarly effective.
Perhaps this isn’t just political expediency – perhaps it’s human nature.
Or perhaps it’s not.
What do you think?
Published in General
But if the person in front of you is lousy at stats, could still be a problem.
This is the George McClellan syndrome. You have more than enough resources and information to take reasonably constructive action but get yourself in trouble by refusing to act with less than perfect information and instead take pointless, costly, self-defeating defensive postures only useful against imaginary threats.
There may have been a time when credentials carried weight as a general rule, but not anymore.
Cholesterol of my 30 year old self just over the line? Statins forever, apparently. Nuts to that. My 47 year old self never having taken the statins? No cholesterol problems. I chose wisely, I think. Certainly I chose more thriftily.
I’d love to be able to find a worthy doctor. Alas, the only one close is my wife’s ob/gyn, but those stirrups don’t fit.
“Susan” may be fighting against what she doesn’t want to believe. Or, she may in fact be both smarter and more invested in her own care than the credentialed person she’s been consulting. I don’t mean you, Doc, but in general. That’s all too real, and I’m coming to believe, the natural state.
Good doctors, like good anything, are rare.
I wrote this long ago when “Nature” was new and scoffed at by the scientific community as a “picture book.”
I don’t recall the exact year I started on statins but let’s say 25 years ago. A few years back the numbers were looking really good but my new cardiologist, I just moved to AZ, went through a spiel about how coming off the statin could be more risky than continuing since I’ve never had any discernible side effects. So there, I’m still taking statins.
🤣🤣🤣
I had a situation similar to Ed G’s earlier this year but regarding elevated blood pressure (hypertension).
I drove to a doctor’s appointment during a snowstorm. I climbed several flights of stairs to see the physician and my blood pressure was checked. I won’t go into what the appointment was actually for, but it wasn’t for blood pressure. My blood pressure had always been below 120/80.
This time, though, my blood pressure was 151/86. The physician didn’t say anything at the time. But a day later I heard from a member of the physician’s staff. She said that she, not the phyisican, raised an alarm about my blood pressure and convinced the physician to have me take a low dose anti-hypertensive until I could follow up with my primary care physician.
I thought this was a clumsy way of diagnosing hypertension (high blood pressure), by taking a single blood pressure measurement in a high stress situation. So, I told this staff member that I would just check my blood pressure every day until I saw my primary care physician and then let this primary care physician decide if my numbers looked good.
I messaged my primary care physician and told him my plan, but he punted the issue back to the physician-specialist I had just seen, the one who wanted me to take the low dose hypertensive.
So, I found a different primary care physician and set up an appointment. This new doctor thought that blood pressure numbers were fine (they were consistently under 120/80). I sent this information to the physician-specialist. Everyone agreed that I didn’t need to take an anti-hypertensive.
Geez.
Statins are the evil medicine de jour. I have less than a 50% success rate at even getting patients with Familial Heterozygous hypercholesterolemia to take them. A recent paper showed that the reported side effects of statins in almost all patients are due to a nocebo effect–that is, a noxious placebo effect–in the perception of the patient but not otherwise reality based. They are one of the safest medicines ever discovered, and have contributed to the reduction of cardiovascular disease over the last 30 years or so. Still, we have supposed scientists writing books trashing statins as showing limited benefit, with very low absolute reduction of cardiovascular events vs relative reduction of events. Vaccine deniers are almost insignificant compared to Statin deniers. And they have far better grounds for their dissent.
It is true that there has never been a really good large, sufficiently long term, sufficiently high dose statin study to establish beyond question the full absolute benefits vs risks of statins. And every patient I see has a friend or relative that supposedly has had severe side effects or injury from statins, and almost all patients are reluctant to take them, so potent are the internet “data” on statins.
Most of the patients that I talk in to trying ezetimibe in place of a statin, report exactly the same side effects from this medicine that they report from statins. Some even report the same side effects from Welchol.
The I offer PCSK-9 inhibitors to those who qualify for these (LDL around 200 or above). Then they completely freak out. Mostly. And it’s rare to get the prior auth through the insurance company anyway, particularly for someone who has not already had a heart attack.
All of that said, John Ionides, the epidemiologist from Stanford, notes that 95% or more of the medical literature is incorrectly analyzed statistically. So even the medical researches get it mostly wrong. And then there is the problem of politicization of medical journals. They have long been corrupt. Witness the article on vaccines causing autism–concocted, fraudulent data, and it took the Lancet more than a decade to retract the paper.
So, we have “evidenced-based” medicine, which is almost an oxymoron. Then we have “eminence-based” medicine, going by the recommendations of the gurus–a fraught approach indeed. Then we have “faith based” medicine, which, by default, is what most of us actually practice out here in the boondocks.
Statistics? Statistics? We don’t need no stinking’ statistics! (To borrow dialogue from The Treasure of the Sierra Madre).
I can give you studies that, while statistically impressive, were completely bogus.
Has any doctor ever actually looked at Lipoprotein(a) as an indicator for heart disease, heart attack and stroke?
Yep. Important marker.
This comment and response makes me think of my relationship to expertise.
Doctors are presumably expert in their respective specialties, and broadly knowledgeable about matters of health. It’s probably unwise for most of us who aren’t doctors to assume that doctors don’t take doctoring seriously, that they make obvious and clumsy mistakes that even non-doctors can readily see. (Of course, if I keep hearing that doctors are numerically illiterate, I might have to revise that view.)
I’ve never been on a movie set, and I don’t know what it’s like, the pressure and dynamic and roles and duties and responsibilities. I know a lot about guns — am even somewhat expert on the topic — and have strong opinions about guns, but that doesn’t include strong opinions about guns used as props on movie sets.
I don’t know beans about philosophy, but somehow feel free to express criticism of the views of professional philosophers, as I did a few minutes ago on SA’s post about this Plantinga fellow. So I’m doing something there that I’d rarely consider doing regarding medicine or movie making.
I suppose that means that I have a fair amount of respect for doctors (who knew?), and more respect for the people who make movies than for the people who write philosophy textbooks.
Huh.
I’m surprised. I’ve always had to ask for it for more than a decade now. But they routinely do a lipid panel.
Yeah, lots of docs ignore it.
Remember, I’m a lipid geek.
Lipid Geek was a garage punk band in Bristol were all the guys knew each other from fastfood work.
Not that there’s a point to this question, but do you routinely order it on an initial visit? Or is it a follow-up test for specific symptoms or prior lab results?
Depends on the situation. Lots and lots and lots of variables…
Lots of Variables was a Techno Band.
and Confounding Variables was ska/punk.
Actually, the impact of statins is much better than this in patients with type 2 diabetes. Plus, cardiologist have an intrinsic conflict of interest, in that they are happy to do stunting to prevent/delay heart attacks. Before there were eluting stents, some 30 years ago, a study was published in the NEJM by cardiologists regarding patients with stable angina. The study consisted of doing treadmills on the patients to see how far they could go without chest pain. Then the patients were randomized to medical management vs angioplasty. After 6 months, the patients were studied again with treadmill stress tests, and the patients who had had angioplasty could go further on the treadmill than those on medical therapy. There was no difference in MI’s or other acute events. The conclusion was that stable angina should be managed by angioplasty. The problem was that the peak incidence of restenosis and acute myocardial infarction after angioplasty occurred at 1 year. That was known at the time of the study. So the study was designed specifically to avoid the bad outcomes that occurred at 1 year. A completely dishonest study. But it established the care standard of performing angioplasties on stable angina.
That standard is now gone.
Further, cardiovascular surgeons used to tell patients that CABG would save their lives. It wasn’t true. The procedure was palliative, resolved angina, and didn’t prolong life at all, except in a small subset of the patients.
Cardiology seems like magic to me. Somebody once looked at squiggles on a tape that to me looked exactly like the squiggles in the picture of normal lines and squiggles on the internet. Sent me for further testing which was OK because the cardiologist was exceptionally good-looking. So, naturally, the one time I actually looked forward to extensive follow-up, it turned out to be nothing.
I went in for liver tests a couple of times. Both times, the very attractive PA wore low cut tops and spent a lot of time bending over. I think she just wanted to give us old guys a thrill.
Sounds like a heart and blood pressure test to me. 😆
Yeah, raise your levels.
Available at Barnes and Noble!
Way up in the early part of this comment thread, I recounted a story of the frustration of the faculty members who were teaching the statistics course to MDs at the university where I eventually got my PhD.
It was sort of a slam on doctors, and I want to make the point that I really appreciate MDs like Dr. Bastiat who work on understanding statistics. I studied (and taught) statistics for years before I fully understood how subtle and elusive a process it is. If you are truly interested in the mysteries of statistics, and willing to spend money and time for that knowledge, I recommend the book Uncertainty: The Soul of Modeling, Probability and Statistics by William Briggs. The book is not for the faint hearted, but it is amazingly perceptive. Briggs wants to revolutionize statistics, and he has the analytical chops to make his case.
What we have now are software packages that make it trivially easy to run a statistical test, but are of no help at all of telling you whether or not the test is useful or whether a perceived difference is important. And powerful stats software combined with data collection and massive databases make it trivially easy to to do bogus studies by just plowing through data and tests until you get a result that crosses the magical P<0.05 threshold that makes it publishable.
This has become such a serious problem in medical research that there are serious efforts to replace the standard criteria for significance with new ways of showing that a result matters. But there is no consensus on how that will work out.
What I tell anybody who asks is that when you read a “popular” news story with the headline or the lead sentence “Studies have shown ….” you might as well skip to the next thing to read.
Anyway, kudos to Dr. Bastiat for his work to understand complex processes to make life better for his patients.
In a book I read years ago, I came across a story about a student in medical school who created a video on how to do CPR. He wanted it to be viewed by the general public as much as possible. So he called it “French Maids.” :-) :-)
That really shouldn’t get past the reviewers, though. I knew a sociology professor who ran a diploma mill for masters level teacher students going for their salary increment, who advised his students to do exactly what you describe when it came to analyzing survey questionnaire responses. When I tried to suggest, very carefully, that it wasn’t quite kosher to do those multiple comparisons without making adjustments, he played dumb. Well, those theses weren’t going to get published anywhere, but I would disappointed in a medical journal that would let something like that slide by.
Unfortunately it does. The case of Brian Wansink at Cornell tells you all you need to know.