Quote of the Day – Warnings and Accidents

 

NASA management had many, many warnings that there was something wrong, but the warnings were disregarded.  It was like a child that runs in the road and the parent is very upset and says it’s very dangerous. Sooner or later the child gets run over. Is it an accident? No it’s not an accident. – Richard Feynman on the Challenger accident.

I was at my office when the Challenger crashed. I had just returned from a meeting at Johnson Space Center with a co-worker.  We took a break to watch the launch – and watched live as the Shuttle program I had been working on for seven years fell apart in flames.

I followed the subsequent investigation closely. All of us did. Eventually the cause was tracked down to a series of events: A leak in the solid rockets let flame escape. It melted the strut holding the solid rocket booster to the external tank. The SRB swung into the external tank, rupturing it. The external tank collapsed, throwing the orbiter off the stack. This happened when the Shuttle was going through maximum dynamic pressure. The wind tore the orbiter apart when the orbiter tilted back as it flew off the tank. There was no explosion. The fireball was the result of hydrogen escaping the tank and igniting when it reached the open flame of the main engines.

Most people missed the real cause of the disaster. Challenger was destroyed through a specific set of conditions and the subsequent series of events that resulted. Yet – as Feynman realized – that was not real reason the Shuttle crashed. The crash was the result of the real reason.  The real reason was that NASA got careless.

Between July 1985 and January 1986, NASA flew seven Shuttle missions.  I was in Mission Control for six of them. (51-L was the first mission since STS-4 I did not support.) In each of those seven missions a problem occurred, which could have caused a catastrophic failure that could have killed a crew. In one mission, sensors indicated the engines were overheating. (It resulted in an abort-to-orbit.) In another, there was a failure in the reaction control system that eliminated redundancy. (Mission rules called for the mission to be terminated and brought home early. It wasn’t.) In another, there was a weather scrub at the transatlantic abort landing strips. The Shuttle launched anyway.

The Challenger flight rules stated you should not launch if the Orbiter was chilled below 30 degrees F. (That was why there was no data on the effects of chilling the SRBs to 27 degrees F. NASA did not pay for the analysis because it was unneeded. You could not launch below 30 degrees due to Orbiter constraints.)

The bigger point is that NASA ran seven straight missions in which a known flaw could have brought down a Shuttle. But they got away with it the first six times. And why wouldn’t they?  The chance that any one flaw would be fatal was miniscule. The seventh proved if you do something stupid enough times, it eventually catches you. Moreover, within the context of each individual system, the hazard was known and could be mitigated. The problem was that hazards had effects outside the individual system.

The SRB leak was an example. SRBs had leaked on earlier missions.  The leak reduced thrust to orbit but otherwise did not matter. You ended up in a lower orbit. Even if the flame played on the external tank, it did not matter. The tank was filled with liquid hydrogen which would absorb the heat of the flame and prevent the tank surface from melting. It only mattered when it melted the strut holding the tank and booster apart.  Of course, then it really mattered.

Feynman realized there was no one cause for the Shuttle crash. If you get sloppy, and ignore flight rules, eventually some combination of circumstances will trip you up.

I bring this up because much the same principle applies to the mid-air collision in Washington, DC. People are focusing on the cause of the accident rather than the reason it happened. Was the helicopter pilot flying too high? Did the helicopter pilot focus on the wrong aircraft? Were there too few air traffic controllers on duty? Were the controllers competent?  There is a belief that if we can identify the cause we will avoid future accidents.

Except . . . we did that with Challenger.  And 16 years later, almost to the day, we lost Columbia to a similar set of foul-ups. People make mistakes.

If the helicopter pilot was 150-200 feet higher than the pilot should have been, that was a mistake. Under normal circumstances, it is not a major error. Similarly, if an air traffic controller was too tired or too busy to make one more check to ensure the helicopter had the right aircraft in sight, the ATC made a mistake. But under normal circumstances, it was not a major error. Yet, in this case, a set of minor mistakes led to tragedy.

The solution is not to insist people not make minor mistakes. Those are going to happen. The solution is to design a system where minor mistakes will not cause a catastrophe. In this case, routing helicopters along the Potomac River perpendicular to the flight path of jetliners landing at (or taking off from) a commercial airport seems a singularly bad idea. It maximizes the opportunity to let a series of minor errors cascade into disaster. Shortly before I wrote this, the DOD announced military helicopters were not to fly along the Potomac near Reagan Field.

Yet it is not as straightforward as simply banning flights. Chesterton’s fence applies. There was a reason for the helicopter flights. This helicopter was on a training mission.  The mission was a nighttime exfiltration of Federal leaders from Washington, DC.  It is the type of thing that may be necessary when the balloon goes up. It is one of those missions you have to get right, and get right in adverse conditions. This requires practice. Lots of practice. Over and over. Our armed forces train like they fight.

The question now comes down to whether there was another way to conduct this mission.  Is there a route the helicopters can fly without flying across approach patterns, and still do this mission as well as before? If the balloon goes up and leaders of the Federal government have to be evacuated under wartime conditions will that mission be completed as efficiently as it would have been using the old flight path? If not, is the loss of efficiency acceptable? As Thomas Sowell once observed, “There are no solutions. There are only trade-offs.”

Trade-offs also apply when it comes to Reagan Airport. The military could fly its original flight profile if Reagan were shut down and the land repurposed. It is a legacy airfield with many problems. There are two other airports close at hand. Yet shutting it down creates other issues. It is convenient. People rely on it. Again, Chesterton’s fence comes into play.

Similarly, trade-offs were the reason NASA flew seven Shuttle missions after flight rules called for the missions to be delayed or cut short. There was a cost to delaying or terminating a mission. Six times that cost proved higher than the benefit gained by continuing. The seventh time, the cost of ignoring flight rules proved higher – and more painful – than an extra two-day’s delay would have been.  The problem was that the analysis done in making the go/no-go decision was flawed. The risks had not been evaluated properly.

The same thing happened on January 29. Several people made flawed risk assessments, with tragic consequences. The real lesson from this tragedy is the same lesson that Challenger offered. People make mistakes. The only way to mitigate those mistakes is to ensure that mistakes, especially small, usually inconsequential mistakes, do not have fatal consequences. In this case, the result of a cascade of mistakes is 67 dead.

Yes, the control tower should have been fully staffed, and with competent people. Yes, the helicopter pilot should have been more concerned with flying too high than flying too low. Yes, people designing the mission flown by the helicopter should have been more aware of outside factors (other than optimizing the mission) than they were. I am sure when the NTSB investigation is done there will be a list of findings as long as my arm.

We will fix the problems that led to this crash going forward. Just like they fixed the problems that led to the Air Florida crash in 1982. There has not been a similar accident since then. There will not be until the Air Florida crash is forgotten. But I can guarantee, there will be another air catastrophe in the future. Like the Flight 5342 crash, it will be caused by a cascading step of minor mistakes that lead to disaster. Because people make mistakes.

Published in Group Writing
This post was promoted to the Main Feed at the recommendation of Ricochet members. Like this post? Want to comment? Join Ricochet’s community of conservatives and be part of the conversation. Join Ricochet for Free.

There are 20 comments.

Become a member to join the conversation. Or sign in if you're already a member.
  1. Dr. Bastiat Member
    Dr. Bastiat
    @drbastiat

    Outstanding.  Thanks.

    • #1
  2. Richard Easton Coolidge
    Richard Easton
    @RichardEaston

    Thanks for this post. STS-27 was almost destroyed due to tile damage. It was a confidential military mission and the astronauts were only allowed to show the ground encrypted videos which did not show the full extent of the damage. NASA almost lost another Shuttle in the second mission after Challenger.

    https://www.americaspace.com/2018/12/09/dying-all-tensed-up-30-years-since-the-troubled-secret-mission-of-sts-27/

    • #2
  3. DonG (¡Afuera!) Coolidge
    DonG (¡Afuera!)
    @DonG

    Great post.   They say that every item on the pre-flight checklist is the result of previous tragedy.  Maybe not literally, but the sentiment is right.  But the checklist cannot be too long or will not be followed with sufficient diligence, thus there are trade-offs even in the checklist.

    I worked on the Shuttle program for a short while.  I can attest to the amount of redundancy in the systems.  The goal for the Shuttle was to be two-fault tolerant, but not all know failures can be made safe.  Trade-offs. Fun fact, each of the Shuttles was also unique, like the same model car made in different years.

    Humans are the weak link in most systems, but also the only part of the system that can create and innovate.  Fun fact #2, the Shuttle fight can be completely autonomous except deploying the landing gear.  Pilots gotta be pilots.

     

     

    • #3
  4. Percival Thatcher
    Percival
    @Percival

    Dr. Bastiat (View Comment):

    Outstanding. Thanks.

    Indeed.

    Another detail of the tragedy is that the tower had requested that the CRJ700 use Runway 33 rather than Runway 01. Runway 33’s approach intersects the helicopter corridor. Another hole in the Swiss cheese.

    • #4
  5. Miffed White Male Member
    Miffed White Male
    @MiffedWhiteMale

    Great post and analysis.

    I have one minor quibble, not directly applicable to this flight.

    Similarly, trade-offs were the reason NASA flew seven Shuttle missions after flight rules called for the missions to be delayed or cut short. There was a cost to delaying or terminating a mission. Six times that cost proved higher than the benefit gained by continuing. The seventh time, the cost of ignoring flight rules proved higher – and more painful – than an extra two-day’s delay would have been.  The problem was that the analysis done in making the go-no go decision was flawed. The risks had not been evaluated properly.

    Flight rules like that are made in a  vacuum.  They have value, and should not be ignored without reason.

    But…

    Apollo 12 was struck by lightning 35 seconds into its flight, scrambling the electronic and guidance systems.  Through some fast work  they made it safely to orbit.

    Now, there’s no way a pre-launch flight rule would have been written to say “If the space craft gets struck by lightning during launch and we have all kinds of electric and guidance system problems, we’ll do some quick checks and then three hours later we’ll go ahead and execute the burn to proceed to the moon.”

    But it was decided since they’d gotten safely through one of the most dangerous parts of the mission, the launch to orbit, the risk/benefit ration weighed in favor of proceeding.

    • #5
  6. Seawriter Contributor
    Seawriter
    @Seawriter

    DonG (¡Afuera!) (View Comment):
    Fun fact #2, the Shuttle fight can be completely autonomous except deploying the landing gear.  Pilots gotta be pilots.

    There was an autoland for the Orbiter after Columbia. It was supposed to be tested on the second flight following return to flight.  Yet on that flight – and every subsequent flight – the spacecraft commander aborted autoland and landed manually. Autoland was never used. Crews had a different excuse justification on each landing, but the real reason was the commander and pilot were unwilling to pass up what was usually a once-in-a-lifetime opportunity to land a spaceplane. NASA did not feel like calling the crew on this, and besides, in most cases the commander was flying his or her last mission, so there wasn’t anything NASA could really do to them.

    • #6
  7. Miffed White Male Member
    Miffed White Male
    @MiffedWhiteMale

    Seawriter (View Comment):

    DonG (¡Afuera!) (View Comment):
    Fun fact #2, the Shuttle fight can be completely autonomous except deploying the landing gear. Pilots gotta be pilots.

    There was an autoland for the Orbiter after Columbia. It was supposed to be tested on the second flight following return to flight. Yet on that flight – and every subsequent flight – the spacecraft commander aborted autoland and landed manually. Autoland was never used. Crews had a different excuse justification on each landing, but the real reason was the commander and pilot were unwilling to pass up what was usually a once-in-a-lifetime opportunity to land a spaceplane. NASA did not feel like calling the crew on this, and besides, in most cases the commander was flying his or her last mission, so there wasn’t anything NASA could really do to them.

     

    • #7
  8. Seawriter Contributor
    Seawriter
    @Seawriter

    Miffed White Male (View Comment):

    Now, there’s no way a pre-launch flight rule would have been written to say “If the space craft gets struck by lightning during launch and we have all kinds of electric and guidance system problems, we’ll do some quick checks and then three hours later we’ll go ahead and execute the burn to proceed to the moon.”

    But it was decided since they’d gotten safely through one of the most dangerous parts of the mission, the launch to orbit, the risk/benefit ration weighed in favor of proceeding. 

    Well, yeah. There was no flight rule covering a lightning strike during Apollo 12. (There was on subsequent Apollo missions.) So there was no flight rule to ignore or hand-wave away. So they did an analysis real-time on-orbit. (Also there was a set of protocols to cover the results of a lighting strike, which were implemented immediately and recovered the electrical system.

    Doing analysis on the fly is a lot different than disregarding an existing flight rule that was instituted after a lot of preflight analysis. Chesterton’s fence really does apply to existing flight rules. “Can you something bad is going to happen to the SRBs if we launch at 28 degrees when the flight rules say you should not launch if the Orbiter is chilled to 32 degree?” is a different proposition than “oh gosh, we got hit by lighting during ascent” and you have two revs (three hours) to check out all the systems to see they were working. Chesterton’s fence really did apply in the first case, but no one was able to pursue the reason as to the flight rule (it was an Orbiter rule rather than an SRB rule) in the time available.

    • #8
  9. kedavis Coolidge
    kedavis
    @kedavis

    Seawriter:

    Yet it is not as straightforward as simply banning flights. Chesterton’s fence applies. There was a reason for the helicopter flights. This helicopter was on a training mission.  The mission was a nighttime exfiltration of Federal leaders from Washington D. C.  It is the type of thing that may be necessary when the balloon goes up. It is one of those missions you have to get right, and get right in adverse conditions. This requires practice. Lots of practice. Over and over. Our armed forces train like they fight.

    The question now becomes is there another way to conduct this mission?  Is there a route the helicopters can fly without flying across approach patterns, and still do this mission as well as before? If the balloon goes up and leaders of the Federal government have to be evacuated under wartime conditions will that mission be completed as efficiently as it would have been using the old flight path? If not, is the loss of efficiency acceptable? As Thomas Sowell once observed, “There are no solutions. There are only trade-offs.”

    Trade-offs also apply when it comes to Reagan Airport. The military can fly its original flight profile if Reagan were shut down and the land repurposed. It is a legacy airfield with many problems. There are two other airports close at hand. Yet shutting it down creates other issues. It is convenient. People rely on it. Again, Chesterton’s fence comes into play.

    Or, since I expect the actual exfiltration missions would not be flying when civilian activity is at full throttle, how about occasionally shutting down Reagan for maybe a couple hours, a couple times a month or whatever, for these practices?

    • #9
  10. Jimmy Carter Member
    Jimmy Carter
    @JimmyCarter

    “Our armed forces train like they fight.”

    Our armed forces fight like They train.

    • #10
  11. Percival Thatcher
    Percival
    @Percival

    Jimmy Carter (View Comment):

    “Our armed forces train like they fight.”

    Our armed forces fight like They train.

    The goal of the Roman legions was to have actual combat be an exceptionally bloody training drill.

    • #11
  12. doulalady Member
    doulalady
    @doulalady

    Very useful and enlightening thread guys. Thank you. 
    It strikes me, as a complete ignoramus about the resources available to the air traffic controller, that he was put in the metaphorical position of rubbing his tummy while patting his head. One aircraft was traveling in a mostly ( given the winds) straight line horizontally, while the other was traveling in a mostly (given the winds) straight line vertically. Just one fraction of a degree difference in either trajectory, or one gust of wind hitting either aircraft, and it would have been a near miss. What were the chances?

    • #12
  13. kedavis Coolidge
    kedavis
    @kedavis

    doulalady (View Comment):

    Very useful and enlightening thread guys. Thank you.
    It strikes me, as a complete ignoramus about the resources available to the air traffic controller, that he was put in the metaphorical position of rubbing his tummy while patting his head. One aircraft was traveling in a mostly ( given the winds) straight line horizontally, while the other was traveling in a mostly (given the winds) straight line vertically. Just one fraction of a degree difference in either trajectory, or one gust of wind hitting either aircraft, and it would have been a near miss. What were the chances?

    Or maybe there WAS a gust of wind, that wound up CAUSING the collision.

    Six of one…

    • #13
  14. Seawriter Contributor
    Seawriter
    @Seawriter

    doulalady (View Comment):
    What were the chances?

    Very, very small, individually. But with enough opportunities the odds of a very low probability event occurring grows and grows.

    • #14
  15. EODmom Coolidge
    EODmom
    @EODmom

    Seawriter (View Comment):

    doulalady (View Comment):
    What were the chances?

    Very, very small, individually. But with enough opportunities the odds of a very low probability event occurring grows and grows.

    And we are considering only those individual events or facts about which we know. There really are a lot of unknowns. I’ve been reflecting on all the (sort of) related facts our Marine has told us over the year about training and certification events (if that’s what this helicopter mission was.) There’s not a lot that fits the daily life experience of most of us. The most recent Marine EOD deaths were not in combat – they were at the end of a range day. (2 other Navy techs died in Afghanistan before the withdrawal.) 

    • #15
  16. Instugator Thatcher
    Instugator
    @Instugator

    kedavis (View Comment):
    Or, since I expect the actual exfiltration missions would not be flying when civilian activity is at full throttle, how about occasionally shutting down Reagan for maybe a couple hours, a couple times a month or whatever, for these practices?

    Reagan has reduced operating hours from 10PM to 7AM to comply with noise abatement rules.

    The crash occurred at  around 9 PM.

    • #16
  17. Instugator Thatcher
    Instugator
    @Instugator

    DonG (¡Afuera!) (View Comment):
    Pilots gotta be pilots.

    You can be dang sure that if NASA is gonna make me practice landing that thing 10,000 times in the simulator, I will for sure be landing it for reals on the 10,001 time.

    Pilots gotta pilot.

    • #17
  18. BillJackson Coolidge
    BillJackson
    @BillJackson

    This is a great post and why I came back to Ricochet. Thank you for sharing it.

    In particular because earlier news reports had made it sound like the primary reason for the flights was to shuttle VIPs … and kinda sorta implied that such flights were a perk so our leaders didn’t have to get stuck in traffic. What you describe makes more sense.

    [It also spared all of Ricochet a half-baked post of mine. Something like, “Golly, if the rest of us have to sit in traffic, why don’t our representatives? Shouldn’t they bear the consequences of their decisions? If they’re going to call it public ‘service’… etc, etc.”]

    Finally, as you point out, there are tradeoffs in everything. I lead a software development team and so much of my job is saying “We can give you this new thing you want, but the tradeoff is this other thing will be delayed …” It’s something nobody — including myself — really wants to face. And we tend to accept the tradeoffs until something bites us in the butt, at which point we look for someone/something to blame.

    • #18
  19. Sisyphus Member
    Sisyphus
    @Sisyphus

    kedavis (View Comment):

    Seawriter:

    Yet it is not as straightforward as simply banning flights. Chesterton’s fence applies. There was a reason for the helicopter flights. This helicopter was on a training mission. The mission was a nighttime exfiltration of Federal leaders from Washington D. C. It is the type of thing that may be necessary when the balloon goes up. It is one of those missions you have to get right, and get right in adverse conditions. This requires practice. Lots of practice. Over and over. Our armed forces train like they fight.

    The question now becomes is there another way to conduct this mission? Is there a route the helicopters can fly without flying across approach patterns, and still do this mission as well as before? If the balloon goes up and leaders of the Federal government have to be evacuated under wartime conditions will that mission be completed as efficiently as it would have been using the old flight path? If not, is the loss of efficiency acceptable? As Thomas Sowell once observed, “There are no solutions. There are only trade-offs.”

    Trade-offs also apply when it comes to Reagan Airport. The military can fly its original flight profile if Reagan were shut down and the land repurposed. It is a legacy airfield with many problems. There are two other airports close at hand. Yet shutting it down creates other issues. It is convenient. People rely on it. Again, Chesterton’s fence comes into play.

    Or, since I expect the actual exfiltration missions would not be flying when civilian activity is at full throttle, how about occasionally shutting down Reagan for maybe a couple hours, a couple times a month or whatever, for these practices?

    In the event of a real emergency exfilitration of VIPs from Reagan, will they be stopping all flights so that the inconvenienced, possibly on the receiving end of an event the VIPs are trying to avoid, can stop and watch elites steal any chance that they themselves might escape? Or just make them that much more behind schedule?

    • #19
  20. kedavis Coolidge
    kedavis
    @kedavis

    Sisyphus (View Comment):

    kedavis (View Comment):

    Seawriter:

    Yet it is not as straightforward as simply banning flights. Chesterton’s fence applies. There was a reason for the helicopter flights. This helicopter was on a training mission. The mission was a nighttime exfiltration of Federal leaders from Washington D. C. It is the type of thing that may be necessary when the balloon goes up. It is one of those missions you have to get right, and get right in adverse conditions. This requires practice. Lots of practice. Over and over. Our armed forces train like they fight.

    The question now becomes is there another way to conduct this mission? Is there a route the helicopters can fly without flying across approach patterns, and still do this mission as well as before? If the balloon goes up and leaders of the Federal government have to be evacuated under wartime conditions will that mission be completed as efficiently as it would have been using the old flight path? If not, is the loss of efficiency acceptable? As Thomas Sowell once observed, “There are no solutions. There are only trade-offs.”

    Trade-offs also apply when it comes to Reagan Airport. The military can fly its original flight profile if Reagan were shut down and the land repurposed. It is a legacy airfield with many problems. There are two other airports close at hand. Yet shutting it down creates other issues. It is convenient. People rely on it. Again, Chesterton’s fence comes into play.

    Or, since I expect the actual exfiltration missions would not be flying when civilian activity is at full throttle, how about occasionally shutting down Reagan for maybe a couple hours, a couple times a month or whatever, for these practices?

    In the event of a real emergency exfilitration of VIPs from Reagan, will they be stopping all flights so that the inconvenienced, possibly on the receiving end of an event the VIPs are trying to avoid, can stop and watch elites steal any chance that they themselves might escape? Or just make them that much more behind schedule?

    If there is such an emergency, I don’t know that anyone will be very concerned with regular schedules.

    • #20
Become a member to join the conversation. Or sign in if you're already a member.