Cargo Cult Science

 

The first principle [of science] is that you must not fool yourself — and you are the easiest person to fool. –– Richard Feynman, from his 1974 commencement address at Caltech

South Sea Island Infrastructure Project

In 1974, Richard Feynman gave the commencement address at Caltech, in which he cautioned the audience to understand and be wary of the difference between real science and what he called “Cargo Cult” science. The lecture was a warning to students that science is a rigorous field that must remain grounded in hard rules of evidence and proof. Feynman went on to explain to the students that science is extremely hard to get right, that tiny details matter, that it is always a struggle to make sure that personal bias and motivated reasoning are excluded from the process.

It’s not the good will, the high intelligence, or the expertise of scientists that makes science work as the best tool for discovering the nature of the universe. Science works for one simple reason: It relies on evidence and proof. It requires hypothesis, prediction, and confirmation of theories through careful experiment and empirical results. It requires excruciating attention to detail, and a willingness to abandon an idea when an experiment shows it to be false. Failure to follow the uncompromising rules of science opens the door to bias, group-think, politically-motivated reasoning, and other failures.

Science is the belief in the ignorance of experts. — Richard Feynman

As an example of how unconscious bias can influence even the hardest of sciences, Feynman recounted the story of the Millikan Oil Drop Experiment. The purpose of the experiment was to determine the value of the charge of an electron. This was a rather difficult thing to measure with the technology of the time, and Millikan got a result that was just slightly too high due to experimental error — he used the wrong value for the viscosity of air in his calculations. This was the result that was published.

Now, a slightly incorrect result is not a scandal — it’s why we insist on replication. Even the best scientists can get it wrong once in awhile. This is why the standard protocol is to publish all data and methods so that other scientists can attempt to replicate the results. Millikan duly published his methods along with the slightly incorrect result, and others began doing oil drop experiments themselves.

As others published their own findings, an interesting pattern emerged: The first published results after Millikan’s were also high – just not quite as much. And the next generation of results were again too high, but slightly lower than the last . This pattern continued for some time until the experiments converged on the true number.

Why did this happen? There was nothing about the experiment that should lead to a consistently high answer. If it was just a hard measurement to make, you would expect experimental results to be randomly distributed around the real value. What Feynman realized was that psychological bias was at work: Millikan was a great scientist, and no one truly expected him to be wrong. So when other scientists found their results were significantly different from his, they would assume that they had made some fundamental error and throw the results out. But when randomness in the measurement resulted in a measurement closer to Millikan’s, they assumed that it was a better result. They were filtering the data until the result reached a value that was at least close enough to Millikan’s that the error was ‘acceptable’. And then when that result was added to the body of knowledge, it made the next generation of researchers a little more willing to settle on an even smaller, but still high result.

Note that no one was motivated by money, or politics, or by anything other than a desire to be able to replicate a great man’s work. They all wanted to do the best job they could and find the true result. They were good scientists. But even the subtle selection bias caused by Millikan’s stature was enough to distort the science for some time.

The key thing to note about this episode is that eventually they did find the real value, but not by relying on the consensus of experts or the gravitas and authority of a great scientist. No, the science was pulled back to reality only because of the discipline of constant testing and because the scientific question was falsifiable and experimentally determinable.

Failure to live up to these standards, to apply the rigor of controlled double-blind tests, predictions followed by tests of those predictions and other ways of concretely testing for the truth of a proposition means you’re not practising science, no matter how much data you have, how many letters you have after your signature, or how much money is wrapped up in your scientific-looking laboratory. At best, you are practising cargo-cult science, or as Friedrich Hayek called it in his Nobel speech, ‘scientism’ – adopting the trappings of science to bolster an argument while at the same time ignoring or glossing over the rigorous discipline at the heart of true science.

This brings us back to cargo cults. What is a cargo cult, and why is it a good metaphor for certain types of science today? To see why, let’s step back in time to World War II, and in particular the war against Japan.

The Pacific Cargo Cults

During World War II, the allies set up forward bases in remote areas of the South Pacific. Some of these bases were installed on islands populated by locals who had never seen modern technology, who knew nothing of the strange people coming to their islands. They watched as men landed on their island in strange steel boats, and who then began to cut down jungle and flatten the ground. To the islanders, it may have looked like an elaborate religious ritual.

In due time, after the ground was flat and lights had been installed along its length, men with strange disks over their ears spoke into a little box in front of their mouths, uttering incantations. Amazingly, after each incantation a metal bird would descend from the sky and land on the magic line of flat ground. These birds brought great wealth to the people – food they had never seen before, tools, and medicines. Clearly the new God had great power.

Years after the war ended and the strange metal birds stopped coming, modern people returned to these islands and were astonished by what they saw; ‘runways’ cut from the jungle by hand, huts with bamboo poles for antennas, locals wearing pieces of carved wood around their ears and speaking into wooden ‘microphones’, imploring the great cargo god of the sky to bring back the metal birds.

Ceremony for the new Tuvaluan Stimulus Program

Ceremony for the new Tuvaluan Stimulus Program

Understand, these were not stupid people. They were good empiricists. They painstakingly watched and learned how to bring the cargo birds. If they had been skilled in modern mathematics, they might even have built mathematical models exploring the correlations between certain words and actions and the frequency of cargo birds appearing. If they had sent explorers out to other islands, they could have confirmed their beliefs: every island with a big flat strip and people with devices on their heads were being visited by the cargo birds. They might have found that longer strips bring even larger birds, and used that data to predict that if they found an island with a huge strip it would have the biggest birds.

Blinded With Science!

Blinded with Science

There’s a lot of “science” that could have been done to validate everything the cargo culters believed. There could be a strong consensus among the most learned islanders that their cult was the ‘scientific’ truth. And they could have backed it up with data, and even some simple predictions. For example, the relationship between runway length and bird size, the fact that the birds only come when it’s not overcast, or that they tended to arrive on a certain schedule. They might even have been able to dig deeply into the data and find all kinds of spurious correlations, such as a relationship between the number of birds on the ground and how many were in the sky, or the relationship between strange barrels of liquid on the ground and the number of birds that could be expected to arrive. They could make some simple short-term predictions around this data, and even be correct.

Then one day, the predictions began to fail. The carefully derived relationships meticulously measured over years failed to hold. Eventually, the birds stopped coming completely, and the strange people left. But that wasn’t a problem for the island scientists: They knew the conditions required to make the birds appear. They meticulously documented the steps taken by those first strangers on the island to bring the birds in the first place, and they knew how to control for bird size by runway length, and how many barrels of liquid were required to entice the birds. So they put their best engineers to work rebuilding all that with the tools and materials they had at hand – and unexpectedly failed.

How did all these carefully derived relationships fail to predict what would happen? Let’s assume these people had advanced mathematics. They could calculate p-values, do regression analysis, and had most of the other tools of science. How could they collect so much data and understand so much about the relationships between all of these activities, and yet be utterly incapable of predicting what would happen in the future and be powerless to control it?

The answer is that the islanders had no theory for what was happening, had no way of testing their theories even if they had had them, and were hampered by being able to see only the tiniest tip of an incredibly complex set of circumstances that led to airplanes landing in the South Pacific.

Imagine two island ‘scientists’ debating the cause of their failure. One might argue that they didn’t have metal, and bamboo wasn’t good enough. Another might argue that his recommendation for how many fake airplanes should be built was ignored, and the fake airplane austerity had been disastrous. You could pore over the reams of data and come up with all sorts of ways in which the recreation wasn’t quite right, and blame the failure on that. And you know what? This would be an endless argument, because there was no way of proving any of these propositions. Unlike Millikan, they had no test for the objective truth.

And in the midst of all their scientific argumentation as to which correlations mattered and which didn’t, the real reason the birds stopped coming was utterly opaque to them: The birds stopped coming because some people sat on a gigantic steel ship they had never seen, anchored in the harbor of a huge island they had never heard of, and signed a piece of paper agreeing to end the war that required those South Pacific bases. And the signing itself was just the culmination of a series of events so complex that even today historians argue over it. The South Sea Islanders were doomed to hopeless failure because what they could see and measure was a tiny collection of emergent properties caused by something much larger, very complex and completely invisible to them. The correlations so meticulously collected were not describing fundamental, objective properties of nature, but rather the side-effects of a temporary meta-stability of a constantly changing, wholly unpredictable and wildly complex system.

The Modern Cargo Cults

Today, entire fields of study are beginning to resemble a form of modern cargo cult science. We like to fool ourselves into thinking that because we are modern, ‘scientific’ people that we could never do anything as stupid as the equivalent of putting coconut shells on our ears and believing that we could communicate with metal birds in the sky through them. But that’s exactly what some are doing in the social sciences, in macroeconomics, and to some extent in climate science and in some areas of medicine. And these sciences share a common characteristic with the metal birds of the south sea cargo cults: They are attempts to understand, predict, and control large complex systems through examination of their emergent properties and the relationships between them.

No economist can hope to understand the billions of decisions made every day that contribute to change in the economy. So instead, they choose to aggregate and simplify the complexity of the economy into a few measures like GDP, consumer demand, CPI, aggregate monetary flows, etc. They do this so they can apply mathematics to the numbers and get ‘scientific’ results. But like the South Sea islanders, they have no way of proving their theories and a multitude of competing explanations for why the economy behaves as it does with no objective way to solve disputes between them. In the meantime, their simplifications may have aggregated away the information that’s actually important for understanding the economy.

You can tell that these ‘sciences’ have gone wrong by examining their track record of prediction (dismal), and by noticing that there does not seem to be steady progress of knowledge, but rather fads and factions that ebb and flow with the political tide. In my lifetime I have seen various economic theories be discredited, re-discovered, discredited once more, then rise to the top again. There are still communist economics professors, for goodness’ sake. That’s like finding a physics professor who still believes in phlogiston theory. And these flip-flops have nothing to do with the discovery of new information or new techniques, but merely by which economic faction happens to have random events work slightly in favor of their current model or whose theories give the most justification for political power.

As Nate Silver pointed out in his excellent, “The Signal and the Noise,” economists’ predictions of future economic performance are no better than chance once you get away from the immediate short term. Annual surveys of macroeconomists return predictions that do no better than what you’d get throwing darts at a dartboard. When economists like Christina Romer have the courage to make concrete predictions of the effects of their proposed interventions, they turn out to be wildly incorrect. And yet, these constant failures never seem to falsify their underlying beliefs. Like the cargo cultists, they’re sure that all they need to do is comb through the historical patterns in the economy and look for better information, and they’ll surely be able to control the beast next time.

Other fields in the sciences are having similar results. Climate is a complex system with millions of feedbacks. It adapts and changes by its own rules we can’t begin to fully grasp. So instead we look to the past for correlations and then project them, along with our own biases, into the future. And so far, the history of prediction of climate models is very underwhelming.

In psychology, Freudian psychoanalysis was an unscientific, unfalsifiable theory based on extremely limited evidence. However, because it was being pushed by a “great man” who commanded respect in the field, it enjoyed widespread popularity in the psychology community for many decades despite there being no evidence that it worked. How many millions of dollars did hapless patients spend on Freudian psychotherapy before we decided it was total bunk? Aversion therapy has been used for decades for the treatment of a variety of ills by putting the patient through trauma or discomfort, despite there being very little clinical evidence that it works. Ulcers were thought to have been caused by stress. Facilitated communication was a fad that enjoyed widespread support for far too long.

A string of raw facts; a little gossip and wrangle about opinions; a little classification and generalization on the mere descriptive level; a strong prejudice that we have states of mind, and that our brain conditions them: but not a single law in the sense in which physics shows us laws, not a single proposition from which any consequence can causally he deduced. This is no science, it is only the hope of a science.

— William James, “Father of American psychology”, 1892

These fields are adrift because there are no anchors to keep them rooted in reality. In real science, new theories are built on a bedrock of older theories that have withstood many attempts to falsify them, and which have proven their ability to describe and predict the behavior of the systems they represent. In cargo cult sciences, new theories are built on a foundation of sand — of other theories that themselves have not passed the tests of true science. Thus they become little more than fads or consensus opinions of experts — a consensus that ebbs and flows with political winds, with the presence of a charismatic leader in one faction or another, or with the accumulation of clever arguments that temporarily outweigh the other faction’s clever arguments. They are better described as branches of philosophy, and not science — no matter how many computer models they have or how many sophisticated mathematical tools they use.

In a cargo cult science, factions build around popular theories, and people who attempt to discredit them are ostracised. Ad hominem attacks are common. Different theories propagate to different political groups. Data and methods are often kept private or disseminated only grudgingly. Because there are no objective means to falsify theories, they can last indefinitely. Because the systems being studied are complex and chaotic, there are always new correlations to be found to ‘validate’ a theory, but rarely a piece of evidence to absolutely discredit it. When an economist makes a prediction about future GDP or the effect of a stimulus, there is no identical ‘control’ economy that can be used to test the theory, and the real economy is so complex that failed predictions can always be explained away without abandoning the underlying theory.

There is currently a crisis of non-reproducibility going on in these areas of study. In 2015, Nature looked at 98 peer-reviewed papers in psychology, and found that only 39 of them had results that were reproducible. Furthermore, 97 percent of the original studies claimed that their results were statistically significant, while only 36 percent of the replication studies found statistically significant results. This is abysmal, and says a lot about the state of this “science.”

This is not to say that science is impossible in these areas, or that it isn’t being done. All the areas I mentioned have real scientists working in them using the real methods of science. It’s not all junk. Real science can help uncover characteristics and behaviors of complex systems, just as the South Sea Islanders could use their observations to learn concrete facts such as the amount of barrels of fuel oil being an indicator of how many aircraft might arrive. In climate science, there is real value to be had in studying the relationships between various aspects of the climate system — so long as we recognize that what we are seeing is subject to change and that what is unseen may represent the vast majority of interactions.

The complex nature of these systems and our inability to carry out concrete tests means we must approach them with great humility and understand the limits of our knowledge and our ability to predict what they will do. And we have to be careful to avoid making pronouncements about truth or settled science in these areas, because our understanding is very limited and likely to remain so.

Science alone of all the subjects contains within itself the lesson of the danger of belief in the infallibility of the greatest teachers of the preceding generation.

— Richard Feynman

Published in General
Like this post? Want to comment? Join Ricochet’s community of conservatives and be part of the conversation. Join Ricochet for Free.

There are 103 comments.

Become a member to join the conversation. Or sign in if you're already a member.
  1. Seawriter Contributor
    Seawriter
    @Seawriter

    Like! Like. Like Like.

    Seawriter

    • #1
  2. Seawriter Contributor
    Seawriter
    @Seawriter

    Once upon a time The Journal of Irreproducible Results was the title of a satirical magazine about science. Today it is a description for many peer-reviewed scientific journals.

    Seawriter

    • #2
  3. Misthiocracy Member
    Misthiocracy
    @Misthiocracy

    So much like.

    • #3
  4. iWe Coolidge
    iWe
    @iWe

    Magnificent. Absolutely magnificent. I suggest Main Feed!

    (and consider submitting to wattsupwiththat as well)

    • #4
  5. Jamie Lockett Inactive
    Jamie Lockett
    @JamieLockett

    So many likes.

    Dan you are one of the real stars of the Member Feed.

    • #5
  6. 1967mustangman Inactive
    1967mustangman
    @1967mustangman

    Couldn’t agree more.

    • #6
  7. Bigfoot Coolidge
    Bigfoot
    @Bigfoot

    Seawriter:Once upon a time The Journal of Irreproducible Results was the title of a satirical magazine about science. Today it is a description for many peer-reviewed scientific journals.

    Seawriter

    It still exists (http://www.jir.com/). Interestingly it now proclaims itself as a humor magazine. My original copies of the journal do not mention humor – they did not need to. Today it appears that you must explicitly declare a spoof to avoid suckering the elites :-)

    • #7
  8. Vance Richards Member
    Vance Richards
    @VanceRichards

    College graduates, on average, earn more money. So if we let everyone graduate college, everyone will have more money because . . . science!

    • #8
  9. tigerlily Member
    tigerlily
    @tigerlily

    Wow! Great article. Agree with the comment above that this needs to go to the main feed.

    • #9
  10. Dan Hanson Thatcher
    Dan Hanson
    @DanHanson

    Thanks very much, everyone.  Glad you liked it!

    I hesitated for quite a while before publishing this,  as I wasn’t sure I could do justice to the ideas in a short form essay.   I would have preferred to explain in more detail how real science is being done in the field of complex systems, the economy, psychology, and the other fields I mentioned because I don’t want to give the impression that it’s all bunk, but this was already pushing the length limit of a Ricochet article.  So I asserted that real science can be done in these fields,  but didn’t really expound on that.  Maybe a subject for another post.

    • #10
  11. drlorentz Member
    drlorentz
    @drlorentz

    Link to Feynman’s address.

    A better example of science gone wrong is N rays. A physicist in France (in Nancy to be precise, hence the name N rays) by the name of Blondlot thought he’d discovered a new kind of radiation. People were discovering rays (cathode rays, X rays, and others) at the time so he joined in. The trouble was, you had to be very discerning to observe these rays. Some folks were skeptical, in particular American physicist Robert Wood. While visiting Blondlot’s lab, Wood surreptitiously  removed a prism from the apparatus even as Blondlot claimed he could still see the N rays. After that, the number of publications about N rays declined precipitously.

    Millikan may have introduced some bias into his observations but Blondlot just imagined the whole thing. Still, some other labs (especially in France) made similar observations. We learned about both cases in school, N rays being the more infamous.

    • #11
  12. Tom Meyer, Ed. Contributor
    Tom Meyer, Ed.
    @tommeyer

    Dan Hanson:In a cargo cult science, factions build around popular theories, and people who attempt to discredit them are ostracised. Ad-hominem attacks are common. Different theories propagate to different political groups. Data and methods are often kept private or disseminated only grudgingly. Because there are no objective means to falsify theories, they can last indefinitely. Because the systems being studied are complex and chaotic, there are always new correlations to be found to ‘validate’ a theory, but rarely a piece of evidence to absolutely discredit it. When an economist makes a prediction about future GDP or the effect of a stimulus, there is no identical ‘control’ economy that can be used to test the theory, and the real economy is so complex that failed predictions can always be explained away without abandoning the underlying theory.

    Word.

    • #12
  13. Addiction Is A Choice Member
    Addiction Is A Choice
    @AddictionIsAChoice

    Q: Why did God make meteorologists?

    A: To make economists look good.

    Fantastic post, Dan!

    • #13
  14. Dan Hanson Thatcher
    Dan Hanson
    @DanHanson

    Link to Feynman’s address.

    A better example of science gone wrong is N rays.

    Excellent point!  I forgot all about that.  And I should have put that link to Feynman’s address in the post.  I’ll correct that.

    • #14
  15. Higgs Boson Inactive
    Higgs Boson
    @HiggsBoson

    Fantastic essay! Thank you for contributing it.

    • #15
  16. Tom Riehl Inactive
    Tom Riehl
    @TrinityWaters

    This is one of the best essays here, ever!  Great work, Dan.  As an engineer, I know you can’t accomplish work by pushing on a rope, and you explained in colorful detail exactly why.  Like the eminent anonymous said, a masterpiece.

    • #16
  17. Aaron Miller Member
    Aaron Miller
    @AaronMiller

    Fascinating. Thanks.

    One minor criticism: We should refer to “good science” vs “bad science”, rather than “real science” vs “unscientific”. Like any other human endeavor, science is a pursuit that can be performed well or poorly. To deny that poorly conducted scientific studies are scientific at all is to make science an idol for adoration rather than a method of learning.

    Furthermore, even poor science can be useful. As a broken machine can be repaired if its failures are identified and corrected, we might find salvageable information and insights in deeply flawed studies after we identify unproven assumptions, errors of implementation, and mistaken conclusions.

    As you say, peer review is pivotal. But reproduction of experiments requires accurate and imaginative analysis (“imaginative” of potential oversights or potentials) to be useful. It is the variety of competing biases and perspectives among scientists, rather than the lack of biases, that leads to verification.

    • #17
  18. Hartmann von Aue Member
    Hartmann von Aue
    @HartmannvonAue

    anonymous:This is a masterpiece. Thank you.

    anonymous just called your post a masterpiece. That anonymous. And you know what? He’s right.

    • #18
  19. James Gawron Inactive
    James Gawron
    @JamesGawron

    Dan,

    Feynman was just marvelous. He had perhaps the most even-handed simple approach and could express it so very well. When I read his book QED on Quantum Theory I absolutely loved his opening chapter. He made it clear that most people have trouble with Quantum because they can’t let their minds learn how it works. They immediately react that “it can’t be that way” so they don’t even follow how to do a simple actual calculation. He then made it clear that he wanted you to turn your need for a deeper explanation off so he could teach you how to do a simple actual Quantum calculation. I did and in the next few chapters I followed his explanation. He was right (surprise!) it wasn’t that hard to follow the Quantum calculation. It was my natural prejudice to assume “it couldn’t be that way” so I was missing the reality because of the cognitive dissonance.

    Hopefully, we are coming out of the huge “cargo cult” of Man Made Global Warming. Jim Lovelock the creator of the GAIA hypothesis refused to endorse it some years ago (the evidence didn’t support the theory – his own theory) and Michael Mann the arch villain idealogue defender of MMGW has just recently signed on to paper recognizing that the data shows no warming in 20 years.

    The false belief in MMGW has done immense damage and is no joke. Hopefully, we are older but wiser.

    Regards,

    Jim

    • #19
  20. Typical Anomaly Inactive
    Typical Anomaly
    @TypicalAnomaly

    Frames the topic fairly and honestly. It really advanced the conversation for me. Many thanks.

    Wanna bet our general cultural yen for instant gratification is a contributor to the problem that prompted the post? I was reminded of how real science can take time and dogged persistence. Instant answers are no the norm.

    But waiting and patience…they’re so yesterday, dude!

    • #20
  21. JimGoneWild Coolidge
    JimGoneWild
    @JimGoneWild

    Giga-Like.

    I think this applies to Global Climate Change science as-well-as economics.

    Thanks.

    • #21
  22. SpiritO'78 Inactive
    SpiritO'78
    @SpiritO78

    had to goggle phlogiston theory but otherwise got through it unscathed. Great piece!

    • #22
  23. Owen Findy Member
    Owen Findy
    @OwenFindy

    Dan, this is outstanding.

    • #23
  24. I Walton Member
    I Walton
    @IWalton

    Outstanding article,I’m sending it all over the place.  Liked that it ended with economics and climate.  Perfect examples of cargo cult science.

    • #24
  25. RickT Member
    RickT
    @RickT

    Great article, one of the best I have ever read.  As a scientist, this issue has frustrated me but you did a great service explaining it so clearly.  I recommend printing or forwarding to any high school or college teacher you know and have the students read it!

    Thanks!

    • #25
  26. Jamie Lockett Inactive
    Jamie Lockett
    @JamieLockett

    The only criticism I have of Dan Hanson essays is that they are often so well researched, written and argued that I rarely have anything I can add to the discussion. I just kind of sit here nodding and wishing I could extend the discussion.

    • #26
  27. Roberto Member
    Roberto
    @Roberto

    Feynman’s lecture on the scientific method also covered much of this ground and is an excellent book end to this piece.

    • #27
  28. Makai Inactive
    Makai
    @Makai

    The fact I actually work in the air cargo industry made the pleasure of reading this deep and insightful article all the more enjoyable.  Mahalo!

    • #28
  29. Midget Faded Rattlesnake Contributor
    Midget Faded Rattlesnake
    @Midge

    Dan Hanson:If it was just a hard measurement to make, you would expect experimental results to be randomly distributed around the real value.

    Oddly enough, I wouldn’t. They already knew Milikan’s value, and to some degree, trusted it. Milikan’s value was real, if imperfect, information, and that information is hard to discount.

    [W]hen other scientists found their results were significantly different from [Milikan’s], they would assume that they had made some fundamental error and throw the results out.

    Which was not, given what they knew, so very strange. After all, we don’t get to test theories with perfect instruments. The claim, “I trust my instruments so much that, when their data contradicts established results, I believe my instruments and not the established results,” is a pretty big claim.

    Polanyi and ET Jaynes, both working scientists, rightly observed, I think, that most “anomalies” in data are justifiably written off – which is one reasons it’s such a big deal when an anomaly does actually point a truth that prior theory missed.

    The key thing to note about this episode is that eventually they did find the real value, but not by relying on the consensus of experts or the gravitas and authority of a great scientist. No, the science was pulled back to reality only because of the discipline of constant testing and because the scientific question was falsifiable and experimentally determinable.

    True, this is a case where new data rectified established knowledge. But it works both ways – we also rely on established knowledge to interpret new data – and I think it working both ways is a good thing.

    In hindsight, we can easily wish that the convergence to a better value for the oil-drop experiment happened faster, but prospectively, I’m not sure there would have been a way to tell that science wasn’t working exactly as it should have been.

    • #29
  30. Tuck Inactive
    Tuck
    @Tuck

    Bravo.  This is an issue I’ve been thinking about for several years.  You did a masterful job of covering it.

    • #30
Become a member to join the conversation. Or sign in if you're already a member.