Cargo Cult Science

 

The first principle [of science] is that you must not fool yourself — and you are the easiest person to fool. –– Richard Feynman, from his 1974 commencement address at Caltech

South Sea Island Infrastructure Project

In 1974, Richard Feynman gave the commencement address at Caltech, in which he cautioned the audience to understand and be wary of the difference between real science and what he called “Cargo Cult” science. The lecture was a warning to students that science is a rigorous field that must remain grounded in hard rules of evidence and proof. Feynman went on to explain to the students that science is extremely hard to get right, that tiny details matter, that it is always a struggle to make sure that personal bias and motivated reasoning are excluded from the process.

It’s not the good will, the high intelligence, or the expertise of scientists that makes science work as the best tool for discovering the nature of the universe. Science works for one simple reason: It relies on evidence and proof. It requires hypothesis, prediction, and confirmation of theories through careful experiment and empirical results. It requires excruciating attention to detail, and a willingness to abandon an idea when an experiment shows it to be false. Failure to follow the uncompromising rules of science opens the door to bias, group-think, politically-motivated reasoning, and other failures.

Science is the belief in the ignorance of experts. — Richard Feynman

As an example of how unconscious bias can influence even the hardest of sciences, Feynman recounted the story of the Millikan Oil Drop Experiment. The purpose of the experiment was to determine the value of the charge of an electron. This was a rather difficult thing to measure with the technology of the time, and Millikan got a result that was just slightly too high due to experimental error — he used the wrong value for the viscosity of air in his calculations. This was the result that was published.

Now, a slightly incorrect result is not a scandal — it’s why we insist on replication. Even the best scientists can get it wrong once in awhile. This is why the standard protocol is to publish all data and methods so that other scientists can attempt to replicate the results. Millikan duly published his methods along with the slightly incorrect result, and others began doing oil drop experiments themselves.

As others published their own findings, an interesting pattern emerged: The first published results after Millikan’s were also high – just not quite as much. And the next generation of results were again too high, but slightly lower than the last . This pattern continued for some time until the experiments converged on the true number.

Why did this happen? There was nothing about the experiment that should lead to a consistently high answer. If it was just a hard measurement to make, you would expect experimental results to be randomly distributed around the real value. What Feynman realized was that psychological bias was at work: Millikan was a great scientist, and no one truly expected him to be wrong. So when other scientists found their results were significantly different from his, they would assume that they had made some fundamental error and throw the results out. But when randomness in the measurement resulted in a measurement closer to Millikan’s, they assumed that it was a better result. They were filtering the data until the result reached a value that was at least close enough to Millikan’s that the error was ‘acceptable’. And then when that result was added to the body of knowledge, it made the next generation of researchers a little more willing to settle on an even smaller, but still high result.

Note that no one was motivated by money, or politics, or by anything other than a desire to be able to replicate a great man’s work. They all wanted to do the best job they could and find the true result. They were good scientists. But even the subtle selection bias caused by Millikan’s stature was enough to distort the science for some time.

The key thing to note about this episode is that eventually they did find the real value, but not by relying on the consensus of experts or the gravitas and authority of a great scientist. No, the science was pulled back to reality only because of the discipline of constant testing and because the scientific question was falsifiable and experimentally determinable.

Failure to live up to these standards, to apply the rigor of controlled double-blind tests, predictions followed by tests of those predictions and other ways of concretely testing for the truth of a proposition means you’re not practising science, no matter how much data you have, how many letters you have after your signature, or how much money is wrapped up in your scientific-looking laboratory. At best, you are practising cargo-cult science, or as Friedrich Hayek called it in his Nobel speech, ‘scientism’ – adopting the trappings of science to bolster an argument while at the same time ignoring or glossing over the rigorous discipline at the heart of true science.

This brings us back to cargo cults. What is a cargo cult, and why is it a good metaphor for certain types of science today? To see why, let’s step back in time to World War II, and in particular the war against Japan.

The Pacific Cargo Cults

During World War II, the allies set up forward bases in remote areas of the South Pacific. Some of these bases were installed on islands populated by locals who had never seen modern technology, who knew nothing of the strange people coming to their islands. They watched as men landed on their island in strange steel boats, and who then began to cut down jungle and flatten the ground. To the islanders, it may have looked like an elaborate religious ritual.

In due time, after the ground was flat and lights had been installed along its length, men with strange disks over their ears spoke into a little box in front of their mouths, uttering incantations. Amazingly, after each incantation a metal bird would descend from the sky and land on the magic line of flat ground. These birds brought great wealth to the people – food they had never seen before, tools, and medicines. Clearly the new God had great power.

Years after the war ended and the strange metal birds stopped coming, modern people returned to these islands and were astonished by what they saw; ‘runways’ cut from the jungle by hand, huts with bamboo poles for antennas, locals wearing pieces of carved wood around their ears and speaking into wooden ‘microphones’, imploring the great cargo god of the sky to bring back the metal birds.

Ceremony for the new Tuvaluan Stimulus Program

Ceremony for the new Tuvaluan Stimulus Program

Understand, these were not stupid people. They were good empiricists. They painstakingly watched and learned how to bring the cargo birds. If they had been skilled in modern mathematics, they might even have built mathematical models exploring the correlations between certain words and actions and the frequency of cargo birds appearing. If they had sent explorers out to other islands, they could have confirmed their beliefs: every island with a big flat strip and people with devices on their heads were being visited by the cargo birds. They might have found that longer strips bring even larger birds, and used that data to predict that if they found an island with a huge strip it would have the biggest birds.

Blinded With Science!

Blinded with Science

There’s a lot of “science” that could have been done to validate everything the cargo culters believed. There could be a strong consensus among the most learned islanders that their cult was the ‘scientific’ truth. And they could have backed it up with data, and even some simple predictions. For example, the relationship between runway length and bird size, the fact that the birds only come when it’s not overcast, or that they tended to arrive on a certain schedule. They might even have been able to dig deeply into the data and find all kinds of spurious correlations, such as a relationship between the number of birds on the ground and how many were in the sky, or the relationship between strange barrels of liquid on the ground and the number of birds that could be expected to arrive. They could make some simple short-term predictions around this data, and even be correct.

Then one day, the predictions began to fail. The carefully derived relationships meticulously measured over years failed to hold. Eventually, the birds stopped coming completely, and the strange people left. But that wasn’t a problem for the island scientists: They knew the conditions required to make the birds appear. They meticulously documented the steps taken by those first strangers on the island to bring the birds in the first place, and they knew how to control for bird size by runway length, and how many barrels of liquid were required to entice the birds. So they put their best engineers to work rebuilding all that with the tools and materials they had at hand – and unexpectedly failed.

How did all these carefully derived relationships fail to predict what would happen? Let’s assume these people had advanced mathematics. They could calculate p-values, do regression analysis, and had most of the other tools of science. How could they collect so much data and understand so much about the relationships between all of these activities, and yet be utterly incapable of predicting what would happen in the future and be powerless to control it?

The answer is that the islanders had no theory for what was happening, had no way of testing their theories even if they had had them, and were hampered by being able to see only the tiniest tip of an incredibly complex set of circumstances that led to airplanes landing in the South Pacific.

Imagine two island ‘scientists’ debating the cause of their failure. One might argue that they didn’t have metal, and bamboo wasn’t good enough. Another might argue that his recommendation for how many fake airplanes should be built was ignored, and the fake airplane austerity had been disastrous. You could pore over the reams of data and come up with all sorts of ways in which the recreation wasn’t quite right, and blame the failure on that. And you know what? This would be an endless argument, because there was no way of proving any of these propositions. Unlike Millikan, they had no test for the objective truth.

And in the midst of all their scientific argumentation as to which correlations mattered and which didn’t, the real reason the birds stopped coming was utterly opaque to them: The birds stopped coming because some people sat on a gigantic steel ship they had never seen, anchored in the harbor of a huge island they had never heard of, and signed a piece of paper agreeing to end the war that required those South Pacific bases. And the signing itself was just the culmination of a series of events so complex that even today historians argue over it. The South Sea Islanders were doomed to hopeless failure because what they could see and measure was a tiny collection of emergent properties caused by something much larger, very complex and completely invisible to them. The correlations so meticulously collected were not describing fundamental, objective properties of nature, but rather the side-effects of a temporary meta-stability of a constantly changing, wholly unpredictable and wildly complex system.

The Modern Cargo Cults

Today, entire fields of study are beginning to resemble a form of modern cargo cult science. We like to fool ourselves into thinking that because we are modern, ‘scientific’ people that we could never do anything as stupid as the equivalent of putting coconut shells on our ears and believing that we could communicate with metal birds in the sky through them. But that’s exactly what some are doing in the social sciences, in macroeconomics, and to some extent in climate science and in some areas of medicine. And these sciences share a common characteristic with the metal birds of the south sea cargo cults: They are attempts to understand, predict, and control large complex systems through examination of their emergent properties and the relationships between them.

No economist can hope to understand the billions of decisions made every day that contribute to change in the economy. So instead, they choose to aggregate and simplify the complexity of the economy into a few measures like GDP, consumer demand, CPI, aggregate monetary flows, etc. They do this so they can apply mathematics to the numbers and get ‘scientific’ results. But like the South Sea islanders, they have no way of proving their theories and a multitude of competing explanations for why the economy behaves as it does with no objective way to solve disputes between them. In the meantime, their simplifications may have aggregated away the information that’s actually important for understanding the economy.

You can tell that these ‘sciences’ have gone wrong by examining their track record of prediction (dismal), and by noticing that there does not seem to be steady progress of knowledge, but rather fads and factions that ebb and flow with the political tide. In my lifetime I have seen various economic theories be discredited, re-discovered, discredited once more, then rise to the top again. There are still communist economics professors, for goodness’ sake. That’s like finding a physics professor who still believes in phlogiston theory. And these flip-flops have nothing to do with the discovery of new information or new techniques, but merely by which economic faction happens to have random events work slightly in favor of their current model or whose theories give the most justification for political power.

As Nate Silver pointed out in his excellent, “The Signal and the Noise,” economists’ predictions of future economic performance are no better than chance once you get away from the immediate short term. Annual surveys of macroeconomists return predictions that do no better than what you’d get throwing darts at a dartboard. When economists like Christina Romer have the courage to make concrete predictions of the effects of their proposed interventions, they turn out to be wildly incorrect. And yet, these constant failures never seem to falsify their underlying beliefs. Like the cargo cultists, they’re sure that all they need to do is comb through the historical patterns in the economy and look for better information, and they’ll surely be able to control the beast next time.

Other fields in the sciences are having similar results. Climate is a complex system with millions of feedbacks. It adapts and changes by its own rules we can’t begin to fully grasp. So instead we look to the past for correlations and then project them, along with our own biases, into the future. And so far, the history of prediction of climate models is very underwhelming.

In psychology, Freudian psychoanalysis was an unscientific, unfalsifiable theory based on extremely limited evidence. However, because it was being pushed by a “great man” who commanded respect in the field, it enjoyed widespread popularity in the psychology community for many decades despite there being no evidence that it worked. How many millions of dollars did hapless patients spend on Freudian psychotherapy before we decided it was total bunk? Aversion therapy has been used for decades for the treatment of a variety of ills by putting the patient through trauma or discomfort, despite there being very little clinical evidence that it works. Ulcers were thought to have been caused by stress. Facilitated communication was a fad that enjoyed widespread support for far too long.

A string of raw facts; a little gossip and wrangle about opinions; a little classification and generalization on the mere descriptive level; a strong prejudice that we have states of mind, and that our brain conditions them: but not a single law in the sense in which physics shows us laws, not a single proposition from which any consequence can causally he deduced. This is no science, it is only the hope of a science.

— William James, “Father of American psychology”, 1892

These fields are adrift because there are no anchors to keep them rooted in reality. In real science, new theories are built on a bedrock of older theories that have withstood many attempts to falsify them, and which have proven their ability to describe and predict the behavior of the systems they represent. In cargo cult sciences, new theories are built on a foundation of sand — of other theories that themselves have not passed the tests of true science. Thus they become little more than fads or consensus opinions of experts — a consensus that ebbs and flows with political winds, with the presence of a charismatic leader in one faction or another, or with the accumulation of clever arguments that temporarily outweigh the other faction’s clever arguments. They are better described as branches of philosophy, and not science — no matter how many computer models they have or how many sophisticated mathematical tools they use.

In a cargo cult science, factions build around popular theories, and people who attempt to discredit them are ostracised. Ad hominem attacks are common. Different theories propagate to different political groups. Data and methods are often kept private or disseminated only grudgingly. Because there are no objective means to falsify theories, they can last indefinitely. Because the systems being studied are complex and chaotic, there are always new correlations to be found to ‘validate’ a theory, but rarely a piece of evidence to absolutely discredit it. When an economist makes a prediction about future GDP or the effect of a stimulus, there is no identical ‘control’ economy that can be used to test the theory, and the real economy is so complex that failed predictions can always be explained away without abandoning the underlying theory.

There is currently a crisis of non-reproducibility going on in these areas of study. In 2015, Nature looked at 98 peer-reviewed papers in psychology, and found that only 39 of them had results that were reproducible. Furthermore, 97 percent of the original studies claimed that their results were statistically significant, while only 36 percent of the replication studies found statistically significant results. This is abysmal, and says a lot about the state of this “science.”

This is not to say that science is impossible in these areas, or that it isn’t being done. All the areas I mentioned have real scientists working in them using the real methods of science. It’s not all junk. Real science can help uncover characteristics and behaviors of complex systems, just as the South Sea Islanders could use their observations to learn concrete facts such as the amount of barrels of fuel oil being an indicator of how many aircraft might arrive. In climate science, there is real value to be had in studying the relationships between various aspects of the climate system — so long as we recognize that what we are seeing is subject to change and that what is unseen may represent the vast majority of interactions.

The complex nature of these systems and our inability to carry out concrete tests means we must approach them with great humility and understand the limits of our knowledge and our ability to predict what they will do. And we have to be careful to avoid making pronouncements about truth or settled science in these areas, because our understanding is very limited and likely to remain so.

Science alone of all the subjects contains within itself the lesson of the danger of belief in the infallibility of the greatest teachers of the preceding generation.

— Richard Feynman

Published in General
Like this post? Want to comment? Join Ricochet’s community of conservatives and be part of the conversation. Join Ricochet for Free.

There are 103 comments.

Become a member to join the conversation. Or sign in if you're already a member.
  1. drlorentz Member
    drlorentz
    @drlorentz

    Roberto: Feynman’s lecture on the scientific method also covered much of this ground and is an excellent book end to this piece.

    “If it disagrees with experiment, it’s wrong. In that simple statement is the key to science.”

    This is a bit of wisdom that has not been absorbed by climate change modelers. Note the divergence between the models and observations over the last decade or so. The excellent agreement before that time is attributable to the tuning of models to fit the data. Model average: red curve, observations: heavy black curve.

    temps

    Source of graph: IPCC AR5.

    • #31
  2. CJinMadison Member
    CJinMadison
    @CJinMadison

    Dan, this really is a fantastic piece. Thank you.

    I remember watching a Nat Geo. documentary 10 years ago. It was all about whales, and how they were going to die out, because their food source (plankton) were dying out.

    Then, the scientists admitted that their study of the plankton had been imperfect, leaving a flawed understanding of the movement of plankton colonies.

    Then, (I swear) immediately after admitting their lack of knowledge, the main scientist said, (my paraphrase) “But, now that we understand how plankton move, now we are really concerned about the whales because they aren’t growing in the right places.”

    *sigh*

    A massively complex system… producing an imperfect understanding… leading to theories that can never be disproven, only recalibrated and strengthened.

    Keep speaking truth.

    CJinMadison

    • #32
  3. Seawriter Contributor
    Seawriter
    @Seawriter

    Midget Faded Rattlesnake: Polanyi and ET Jaynes, both working scientists, rightly observed, I think, that most “anomalies” in data are justifiably written off – which is one reasons it’s such a big deal when an anomaly does actually point a truth that prior theory missed.

    We lost two Shuttles because “anomalies” in the data were “justifiably” written off. The behavior seen was unexpected, so the observations had to be wrong. Except they were not wrong, and ignoring them created situations where the data could not be ignored. The problem was obvious – in retrospect.

    Back the 1980s the Tethered Satellite System mission was being planned. It involved tying a satellite to the Orbiter by a 20 KM string. I was involved in the early navigation analysis of the flight doing independent verification and validation of the mission control team.

    There analysis indicated the mission could not be flown – you could not navigate the Shuttle. My analysis indicated the mission would be an easy flight to navigate.

    Turns out I was right. Why? I listened to the data. I soon realized the Orbiter-TSS system was orbiting about the CG of the system – 100 meters above the Orbiter. The navigation software filtered out the sensor bias to solve for the actual CG.

    The other team saw the data was behaving anomalously, and decided to force the solution to the CG of the Orbiter – where it propagated badly. They obeyed the received wisdom and failed. It was cargo cult engineering.

    Seawriter.

    • #33
  4. drlorentz Member
    drlorentz
    @drlorentz

    Midget Faded Rattlesnake: In hindsight, we can easily wish that the convergence to a better value for the oil-drop experiment happened faster, but prospectively, I’m not sure there would have been a way to tell that science wasn’t working exactly as it should have been.

    Agreed. The Millikan case is not an egregious example of self-delusion in science. There have been worse. As an aside, revisionists also have complained that Mendel’s data were fudged somewhat. I haven’t followed up but it’s plausible that he threw away some peas that didn’t fit. He did get the right answer, though.

    • #34
  5. Seawriter Contributor
    Seawriter
    @Seawriter

    My experience with the TSS analysis is one reason I so liked this article. The other team did not understand the physics of what they were doing. They were pushing buttons by rote, rather like alchemists following an incantation. Instead of attacking my conclusions, they chose to attack me. Since I was not using MCC software I was incompetent.

    Their own results showed the same behavior as I was getting (if you knew how to plot the tether deploy curve against their “error”) and eventually they looked extremely foolish.  I have been suspicious of appeals to authority ever since. Show me your work. Assume everything you know is wrong, and prove each one of your assumptions is right – then go forward.

    Seawriter

    • #35
  6. Tuck Inactive
    Tuck
    @Tuck

    Seawriter: Assume everything you know is wrong, and prove each one of your assumptions is right – then go forward.

    +1000

    Nothing is more dangerous than something you think is right that is in fact wrong.

    • #36
  7. drlorentz Member
    drlorentz
    @drlorentz

    CJinMadison: A massively complex system… producing an imperfect understanding… leading to theories that can never be disproven, only recalibrated and strengthened.

    Karl Popper’s view was the a theory had to be falsifiable to be considered science. There has been an increase of unfalsifiable theories of late. These are not science.

    • #37
  8. Tuck Inactive
    Tuck
    @Tuck

    “Science advances one funeral at a time.” — Max Planck

    Nothing’s lowered my estimation of human nature more than spending the last few years looking at how some areas of science are done.

    Planck’s famous comment still seemed overly gloomy to me.

    Does Science Advance One Funeral at a Time?” (PDF)

    “We study the extent to which eminent scientists shape the vitality of their fields by examining entry rates into the fields of 452 academic life scientists who pass away while at the peak of their scientific abilities…. Overall, these results suggest that outsiders are reluctant to challenge leadership within a field when the star is alive and that a number of barriers may constrain entry even after she is gone.”

    So Planck was wrong: it’s not even that fast.

    (This is a paper by economists, so take it with a grain of salt.)

    • #38
  9. Tuck Inactive
    Tuck
    @Tuck

    anonymous: …if left to operate unimpeded, it self-corrects.

    LOL.  Just don’t hold your breath!

    • #39
  10. Bryan G. Stephens Thatcher
    Bryan G. Stephens
    @BryanGStephens

    drlorentz

    CJinMadison: A massively complex system… producing an imperfect understanding… leading to theories that can never be disproven, only recalibrated and strengthened.

    Karl Popper’s view was the a theory had to be falsifiable to be considered science. There has been an increase of unfalsifiable theories of late. These are not science.

    Where does that leave String Theory?

    • #40
  11. Z in MT Member
    Z in MT
    @ZinMT

    First, I have to sing praise onto this post. Dan, an excellent essay.

    Second, MFR you Bayesian you ;)

    Third, I really like the focus on economics and social science. These disciplines use the tools of science, but too often what they are using the tools for is not science. Too often it is akin to using a hockey stick to fling dog poo over the fence into your neighbors yard and calling it hockey.

    Fourth, in my PhD research I had the opportunity to make the first accurate measurement of a physical constant (not nearly as important as the charge of the electron). It was the frequency of the 3S0 – 3P0 optical transition in Ytterbium 174 (f = 518,294,025,309,217.8(0.9) Hz uncertainty in parenthesis) which is a beautiful bright pure yellow color. When your work can and will be checked by other scientists you tend to be cautious and we made four measurements at different times before we published. Climate scientists seem to be able to get away with bad predictions with no consequences.

    Fifth, this is a really great post!

    • #41
  12. Dan Hanson Thatcher
    Dan Hanson
    @DanHanson

    Midget Faded Rattlesnake:

    Dan Hanson:If it was just a hard measurement to make, you would expect experimental results to be randomly distributed around the real value.

    Oddly enough, I wouldn’t. They already knew Milikan’s value, and to some degree, trusted it. Milikan’s value was real, if imperfect, information, and that information is hard to discount.

    It shouldn’t be.  You go where the experiment takes you.  Ideally,  you would want to do that experiment without ever knowing the result that Millikan got,  precisely to eliminate this kind of bias.

    This is why we do experiments double-blind.   Bias is insidious and extremely hard to eliminate without careful controls.

    Which was not, given what they knew, so very strange. After all, we don’t get to test theories with perfect instruments. The claim, “I trust my instruments so much that, when their data contradicts established results, I believe my instruments and not the established results,” is a pretty big claim.

    But that’s the whole point of doing replication – starting with the assumption that you don’t know what the correct answer is.  If you already believe the answer so strongly that you are willing to ignore your own instrumentation,  you’re doing it wrong.

    If you are worried about instrumental error,  the proper solution is to test the instruments themselves.   That’s why we do gage R&R studies before we trust the results of measurements.   But what we don’t do is start an experiment with the knowledge of the result we are expecting,  then treat data as suspect if it doesn’t conform to our biases.

    I take your point about blind faith in instruments that do not conform to ‘established results’.  But if you are doing a replication study,  you are not assuming that there is an ‘established result’.  Helping to establish – or invalidate –  the result is precisely why you are doing the experiment.

    Polanyi and ET Jaynes, both working scientists, rightly observed, I think, that most “anomalies” in data are justifiably written off – which is one reasons it’s such a big deal when an anomaly does actually point a truth that prior theory missed.

    This is a matter of context and judgement.  It all depends on what you are studying, the nature of the anomaly, etc.

    In hindsight, we can easily wish that the convergence to a better value for the oil-drop experiment happened faster, but prospectively, I’m not sure there would have been a way to tell that science wasn’t working exactly as it should have been.

    drlorentz is right – this is a minor episode,  and it wasn’t that big a deal.  But the reason why it’s so illuminating is because bias crept in to the science for very human,  but very trivial reasons.   Imagine what bias might exist in a field where your career can be destroyed by deviating from the consensus, or where funding depends on finding the ‘right’ answer.

    • #42
  13. Dan Hanson Thatcher
    Dan Hanson
    @DanHanson

    Bryan G. Stephens:

    drlorentz

    CJinMadison: A massively complex system… producing an imperfect understanding… leading to theories that can never be disproven, only recalibrated and strengthened.

    Karl Popper’s view was the a theory had to be falsifiable to be considered science. There has been an increase of unfalsifiable theories of late. These are not science.

    Where does that leave String Theory?

    In philosophy and mathematics,  until we come up with a way to test it.  String theory is a perfect example of how smart people can create incredibly beautiful models that seem to explain a lot of things – but which may not be remotely true.

    That doesn’t make it useless,  any more than relativity theory was useless until we found experimental evidence for it.   It just makes it an interesting hypothesis.  It will become a real theory when we figure out a way to tell if it’s true or not.

    • #43
  14. drlorentz Member
    drlorentz
    @drlorentz

    Bryan G. Stephens:

    Karl Popper’s view was the a theory had to be falsifiable to be considered science. There has been an increase of unfalsifiable theories of late. These are not science.

    Where does that leave String Theory?

    In trouble. Unless there are predictions that can be checked, it’s not science by Popper’s definition. Lots, but not all, of physicists feel the same way.

    • #44
  15. Midget Faded Rattlesnake Member
    Midget Faded Rattlesnake
    @Midge

    Tuck:

    Seawriter: Assume everything you know is wrong, and prove each one of your assumptions is right – then go forward.

    +1000

    Nothing is more dangerous than something you think is right that is in fact wrong.

    I agree the mistake can be very dangerous  – clearly.

    I would disagree that starting each and every time with the literal assumption that everything you know is wrong, and proving every assumption used (no matter how well-established) before using it, is an efficient way to get stuff done. Moreover, I doubt either of you would take the advice you’re giving literally, either. (There’s plenty of stuff I can re-derive, but re-deriving every time what’s already established, whether it’s a mathematical formula or a well-documented empirical value, like the molar mass of carbon, adds nothing (and especially for a klutz like me, the empirical values I “discover” in the lab are quite likely to be very much worse than those already established!).)

    I think what you mean is that we should continue to be cautious about what we already know, and be willing to listen to anomalous data, rather than reflexively dismissing it. Which is all true. But since we can’t perfectly differentiate in advance between noise and signal, taking some risk of dismissing as noise what is in fact signal isn’t necessarily wrong.

    In one example Polanyi mentioned, where the anomalies he and several other chemists documented in tin were not the sort of anomalies that risked stuff falling out of the sky and people dying, dismissing the anomalies as perhaps some impurity they just couldn’t explain rather than publishing “a new property of tin” seemed like the right thing to do. And indeed, hindsight proved them right: after a few years of the anomalies appearing, the anomalies vanished, never to be seen again to this day.

    Obviously, when valuable stuff like lives and tons of money are at stake, people should be quite cautious about what might happen if they make the wrong judgment. Risk-analysis is important, but is there one universal standard of risk-analysis applicable to every scientific question? I doubt it.

    • #45
  16. drlorentz Member
    drlorentz
    @drlorentz

    Dan Hanson: That doesn’t make it useless, any more than relativity theory was useless until we found experimental evidence for it. It just makes it an interesting hypothesis. It will become a real theory when we figure out a way to tell if it’s true or not.

    Relativity had testable predictions from day one. Einstein proposed three tests (assuming you’re referring to general relativity). The tests came quickly. There was already extant a discrepancy with Newtonian gravitation regarding the precession of the perihelion of Mercury’s orbit. Deflection of starlight was found just four years after Einstein’s paper was published. The gravitational redshift took longer. One of the experimenters was on my thesis committee.

    Strings are in a more primitive state. I don’t think there are any testable predictions, much less any tests yet.

    • #46
  17. Bryan G. Stephens Thatcher
    Bryan G. Stephens
    @BryanGStephens

    Dan Hanson:

    Bryan G. Stephens:

    drlorentz

    CJinMadison: A massively complex system… producing an imperfect understanding… leading to theories that can never be disproven, only recalibrated and strengthened.

    Karl Popper’s view was the a theory had to be falsifiable to be considered science. There has been an increase of unfalsifiable theories of late. These are not science.

    Where does that leave String Theory?

    In philosophy and mathematics, until we come up with a way to test it. String theory is a perfect example of how smart people can create incredibly beautiful models that seem to explain a lot of things – but which may not be remotely true.

    That doesn’t make it useless, any more than relativity theory was useless until we found experimental evidence for it. It just makes it an interesting hypothesis. It will become a real theory when we figure out a way to tell if it’s true or not.

    Oh, I agree. I just like making fun of people that act as if it were true. When they say things like “Well, we can never prove there are other universes” then they are no different than someone trying to prove God. Yet these same people will go on and on about how they are sure there is no God.

    • #47
  18. Tuck Inactive
    Tuck
    @Tuck

    Midget Faded Rattlesnake: …I would disagree that starting each and every time with the literal assumption that everything you know is wrong, and proving every assumption used (no matter how well-established) before using it, is an efficient way to get stuff done….

    You’re right, of course, it’s not.  But given the vagaries of the human mind, the stuff you think you remember correctly may often be misremembered.  I know that’s happened to me often—maybe it’s just me!

    The key is that you’ve always got to keep in the back of your mind, that what you think might be right may be in error.

    • #48
  19. Dan Hanson Thatcher
    Dan Hanson
    @DanHanson

    Z in MT:First, I have to sing praise onto this post. Dan, an excellent essay.

    Thank you very much.

    Second, MFR you Bayesian you ;)

    Third, I really like the focus on economics and social science. These disciplines use the tools of science, but too often what they are using the tools for is not science. Too often it is akin to using a hockey stick to fling dog poo over the fence into your neighbors yard and calling it hockey.

    And notice how the most politicized areas of science are precisely the areas that deal in complex systems and which have unfalsifiable theories and conclusions.   These are perfect fields for abusing ‘science’ in the service of advancing a political agenda.  We don’t see liberals and conservatives lining up against each other over quantum theory or the existence of the Higgs Boson.   Reality has an inconvenient way of ending those debates.

    Fourth, in my PhD research I had the opportunity to make the first accurate measurement of a physical constant (not nearly as important as the charge of the electron). It was the frequency of the 3S0 – 3P0 optical transition in Ytterbium 174 (f = 518,294,025,309,217.8(0.9) Hz uncertainty in parenthesis) which is a beautiful bright pure yellow color. When your work can and will be checked by other scientists you tend to be cautious and we made four measurements at different times before we published.

    That is awesome.  And you’re exactly right – the knowledge that there is a real, objective answer and that other people will be checking your work helps to keep everyone honest.

    Climate scientists seem to be able to get away with bad predictions with no consequences.

    Yep.  And economists.  And nutritionists.  And sociologists.

    I’m a bit hesitant to include climate science in this list,  because there is a lot of climate science that is very much ‘real’ science.   And we certainly have learned a lot about climate systems from climate science.   Where they go wrong is in thinking that such a system can be predicted a hundred years in the future,  or that tuned computer models are a reasonable way of doing so.

    It also seems to me that many climate scientists have a weak appreciation for just how complex the system is and how many hidden feedbacks and other interactions there may be.   Complex systems are full of ‘unknown unknowns’,  and Complex Adaptive Systems like the climate are constantly, well,  adapting.  That puts a limit on how much future behavior can be predicted by modeling past behavior.

    • #49
  20. Bryan G. Stephens Thatcher
    Bryan G. Stephens
    @BryanGStephens

    drlorentz:

    Bryan G. Stephens:

    Karl Popper’s view was the a theory had to be falsifiable to be considered science. There has been an increase of unfalsifiable theories of late. These are not science.

    Where does that leave String Theory?

    In trouble. Unless there are predictions that can be checked, it’s not science by Popper’s definition. Lots, but not all, of physicists feel the same way.

    Makes for good TV though, I guess. And, by invoking endless other universes, you can make sure that humans are in on way special.

    Every time they find a new exoplanet, in a solar system nothing like ours, they go to great pains to ignore that our system seems unique so far. That may just be because what we can detect. Who knows. But scientists seems quite biased ahead of time that we cannot be special, even by accident.

    • #50
  21. Midget Faded Rattlesnake Member
    Midget Faded Rattlesnake
    @Midge

    Dan Hanson:

    Midget Faded Rattlesnake:

    …Which was not, given what they knew, so very strange. After all, we don’t get to test theories with perfect instruments. The claim, “I trust my instruments so much that, when their data contradicts established results, I believe my instruments and not the established results,” is a pretty big claim.

    But that’s the whole point of doing replication – starting with the assumption that you don’t know what the correct answer is. If you already believe the answer so strongly that you are willing to ignore your own instrumentation, you’re doing it wrong.

    Well, when we run a test to calibrate an instrument, aren’t we relying on a test where we already believe the answer so strongly that we’ll call the instrument “bad” if it doesn’t conform? As you say:

    If you are worried about instrumental error, the proper solution is to test the instruments themselves.

    How do we test an instrument without testing it against something?

    Of course, we want to test what our instruments record against multiple other instruments – replication – and test the instruments against very well-established facts or theories – calibration. In the course of all of this, we may be relying on many others to have accurately reported and interpreted what their instruments told them. So we are also trusting other people.

    I think you, Seawriter, Tuck, and I are pretty much in agreement about how all this happens. That there is a mundane element of trust in doing science as well as exciting skepticism is all I’m really pointing out.

    • #51
  22. Dan Hanson Thatcher
    Dan Hanson
    @DanHanson

    drlorentz:

    Dan Hanson: That doesn’t make it useless, any more than relativity theory was useless until we found experimental evidence for it. It just makes it an interesting hypothesis. It will become a real theory when we figure out a way to tell if it’s true or not.

    Relativity had testable predictions from day one. Einstein proposed three tests (assuming you’re referring to general relativity). The tests came quickly. There was already extant a discrepancy with Newtonian gravitation regarding the precession of the perihelion of Mercury’s orbit. Deflection of starlight was found just four years after Einstein’s paper was published. The gravitational redshift took longer. One of the experimenters was on my thesis committee.

    Strings are in a more primitive state. I don’t think there are any testable predictions, much less any tests yet.

    Of course you are correct.   My point, though, was that relativity was a hypothesis until it could be tested, just as string theory is. I totally get your distinction that relativity had testable claims,  whereas string theory did not.  But until those tests were carried out,  Relativity was just a hypothesis.   The difference between them was that General Relativity came with a path to promote it from hypothesis to theory,  whereas string theory, so far, has not (although I seem to recall a talk recently about possible tests for it).

    And to give everyone an example of just how important predictions and tests are in science,  think about the fact that we spent a huge amount of money to test for gravitational waves – a test we couldn’t make for a hundred years after Einstein came up with GR,  and a test for a theory which had already passed every one of the many tests that had been thrown at it.  Nevertheless,  we weren’t about to rest until every prediction was tested, just in case there was some nuance in the theory we didn’t quite understand.

    • #52
  23. Tuck Inactive
    Tuck
    @Tuck

    Dan Hanson: Yep. And economists. And nutritionists. And sociologists.

    My personal interest has been medical research.  That’s where most of the money is spent.

    [Ioannidis] charges that as much as 90 percent of the published medical information that doctors rely on is flawed.”

    Study here.

    His most interesting corollary is:

    “The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.”

    • #53
  24. Dan Hanson Thatcher
    Dan Hanson
    @DanHanson

    drlorentz:

    Roberto: Feynman’s lecture on the scientific method also covered much of this ground and is an excellent book end to this piece.

    “If it disagrees with experiment, it’s wrong. In that simple statement is the key to science.”

    This is a bit of wisdom that has not been absorbed by climate change modelers. Note the divergence between the models and observations over the last decade or so. The excellent agreement before that time is attributable to the tuning of models to fit the data. Model average: red curve, observations: heavy black curve.

    temps

    Source of graph: IPCC AR5.

    This is an exact analogy to the cargo cult in the post meticulously gathering data about the behavior of the cargo planes.

    Imagine if they had used a standard technique that today is used to ‘prove’ that tuned models are correct.   Let’s say they had two people measure information about the planes on alternate days,  and then the ‘scientists’ took one set of information and built a model that predicted when the planes would arrive.   They could then run their model and compare the output to the ‘sequestered’ data.   Surprise!  The model worked.   Of course it did – it was tuned to predict what the planes were doing at the time of the measurements.   So long as they got the relationships and ratios correct,  they could accurately ‘predict’ what happened in the past.

    But if they tried to use that model to predict how many planes would arrive in 1946, well…

    • #54
  25. Robert Dammers Thatcher
    Robert Dammers
    @RobertDammers

    I’m reminded of a story my school Physics teacher told. There was a proposal for a standard unit of conceit, and the the unit the “Kan” had been proposed. However, this was considered as impractical as the Farad – after all, even one thousandth part would still be unbearably boastful and conceited.

    Of course the target was not only the historical figure, but my friend Peter, who, it has to be said, became a fairly distinguished philosopher in later life.

    https://en.m.wikipedia.org/wiki/Peter_Millican

    • #55
  26. Tuck Inactive
    Tuck
    @Tuck

    Dan Hanson: …But if they tried to use that model to predict how many planes would arrive in 1946, well…

    What continuously amazes me about the field of climatology is that there’s an entire branch of mathematics that arose from the impossibility of predicting the weather.

    Has not a single one of these so-called scientists heard of chaos theory?

    • #56
  27. Tuck Inactive
    Tuck
    @Tuck

    Tuck:Has not a single one of these so-called scientists heard of chaos theory?

    My amusing story about chaos theory: when the book came out I went through it and built models in Excel to see the effects.  One of them broke Excel.  I couldn’t get it to work again, had to reinstall it.

    • #57
  28. Belt Inactive
    Belt
    @Belt

    I’ve long thought that the Left’s obsession with Keynesian economics is a perfect expression of the ‘Cargo Cult’ model.  They don’t really understand what makes an economy work, or why it’s successful.  So they look at the results, and see that a ‘good’ economy has lots of spending and consumption and investment.  So they figure that they can have the government make lots of investments, and throw money around to increase spending and consumption, and they can make an economy ‘good.’  It’s the equivalent of building a runway on a primitive island and waiting for plane to magically appear and fly to the future.

    In a similar way, you can argue that people who try to bring democracy to a culture that is not committed to the underlying pillars of western civilization is also showing a ‘cargo cult’ mentality.  They are imposing externalities (like, say, elections) in the hopes that they make the foundations magically appear to validate the results and secure the future.

    • #58
  29. Cyrano Inactive
    Cyrano
    @Cyrano

    Addiction Is A Choice:Q: Why did God make meteorologists?

    A: To make economists look good.

    Fantastic post, Dan!

    I’ll heartily laugh with you, but as a meteorologist myself, I need to point out we’re quite well acquainted with our limitations and sometimes frustrated with the public’s inability to listen when we try to communicate the uncertainty of our predictions.  I should hope that economists share the same attitude and frustration.

    I once testified in a trial on a matter of forensic meteorology.  What was X at place Y at time Z?  My analysis of the situation, reflecting the innumerable complications and utilizing a large number of simulations designed to gauge uncertainty, was that X was 80 ± 12.  The lawyer bristled at my lack of precision.  He wanted decimal places.  So, I amended my statement to read that X was 80.0 ± 12.0.

    • #59
  30. drlorentz Member
    drlorentz
    @drlorentz

    Dan Hanson: And we certainly have learned a lot about climate systems from climate science. Where they go wrong is in thinking that such a system can be predicted a hundred years in the future, or that tuned computer models are a reasonable way of doing so.

    The computer models are central to climate policy. Without predictions about the future there’s no disaster looming that requires action. Take away the models and it takes away their raison d’etre.

    Dan Hanson: It also seems to me that many climate scientists have a weak appreciation for just how complex the system is and how many hidden feedbacks and other interactions there may be.

    They probably never read Edward Lorenz’s paper either. If they had, they would understand that the climate system is chaotic.

    • #60
Become a member to join the conversation. Or sign in if you're already a member.