What Happened with Southwest Airlines?

 

Just so no one has expectations, I don’t know what happened with Southwest Airlines and its disaster over the holidays. But it was a huge disaster concerning the most basic part of their business. It would be as though a restaurant forgot to buy food. It would be like a shoe manufacturer forgetting to provide for laces. It would be like an airline not having airplanes to fly. Oh, wait.

Now, so far as I can tell, there are no public explanations. Except that some pilot, a backbencher probably wanting to score points for the pilots’ union, complained publicly and loudly that Southwest has lost its way by letting accountants run the airline. Does anyone really believe this to be the cause? Anyone besides the lazy and uneducated masses that form the body of j-urinalysts, that is? Having a sudden collapse that prevents flights from being scheduled doesn’t sound like a result of penny-pinching.

It sounds to me a lot more like a computer failure. Did their databases get hacked? Did a server farm catch fire? Honestly, computer scheduling software that has been running for decades doesn’t suddenly stop working all at once on a massive scale at the most important travel time of the year. I’m much more inclined to think this is vandalism, or worse, a criminal attack.

Why is no one asking these questions? Or am I missing the discussion somewhere? To me, this should be front-page news, but instead we have finger-pointing at Buttigieg as though some politician was supposed to run things.

Does anyone here have any insight?

Published in Business
This post was promoted to the Main Feed by a Ricochet Editor at the recommendation of Ricochet members. Like this post? Want to comment? Join Ricochet’s community of conservatives and be part of the conversation. Join Ricochet for Free.

There are 67 comments.

Become a member to join the conversation. Or sign in if you're already a member.
  1. BDB Inactive
    BDB
    @BDB

    Bunsen (View Comment):

    Red Herring (View Comment):

    BDB (View Comment):

    Bunsen (View Comment):
    And now today (1/11), the FAA has grounded all flights due to a “system outage”. Apparently their National Air Mission System (NAS) went belly up. This system notifies planes in the air about emergencies, etc so they can adjust as necessary. Obviously no real explaniation as to what happened but pilots are speculating that the outdated system is reaching its limits.

    Doesn’t look like it. The sun has just risen on the east coast:

    On further digging, it’s pretty sparse. I see a lot of flights in the air that are an hour or more delayed, and I found a flight for Newark diverting to DCA — so far just the one divert in a sample of 20 or so.

    Planes already in the air incoming and outgoing international.

    As usual I might be a little behind

    No, you were right — I think this “shutdown” wound up being more a filter.

    • #31
  2. Bunsen Coolidge
    Bunsen
    @Bunsen

    BDB (View Comment):

    Bunsen (View Comment):

    Red Herring (View Comment):

    BDB (View Comment):

    Bunsen (View Comment):
    And now today (1/11), the FAA has grounded all flights due to a “system outage”. Apparently their National Air Mission System (NAS) went belly up. This system notifies planes in the air about emergencies, etc so they can adjust as necessary. Obviously no real explaniation as to what happened but pilots are speculating that the outdated system is reaching its limits.

    Doesn’t look like it. The sun has just risen on the east coast:

    On further digging, it’s pretty sparse. I see a lot of flights in the air that are an hour or more delayed, and I found a flight for Newark diverting to DCA — so far just the one divert in a sample of 20 or so.

    Planes already in the air incoming and outgoing international.

    As usual I might be a little behind

    No, you were right — I think this “shutdown” wound up being more a filter.

    I do know they said any planes in the air at the time where safe as they had received their NOTAMs before takeoff and ATC would let them know of any new issues.

    • #32
  3. Red Herring Coolidge
    Red Herring
    @EHerring

    BDB (View Comment):

    Red Herring (View Comment):

    BDB (View Comment):

    Bunsen (View Comment):
    And now today (1/11), the FAA has grounded all flights due to a “system outage”. Apparently their National Air Mission System (NAS) went belly up. This system notifies planes in the air about emergencies, etc so they can adjust as necessary. Obviously no real explaniation as to what happened but pilots are speculating that the outdated system is reaching its limits.

    Doesn’t look like it. The sun has just risen on the east coast:

    On further digging, it’s pretty sparse. I see a lot of flights in the air that are an hour or more delayed, and I found a flight for Newark diverting to DCA — so far just the one divert in a sample of 20 or so.

    Planes already in the air incoming and outgoing international.

    Most of these were domestic in the 40 or so I sampled. Easily half were delayed, many by over an hour, a handful by several hours. Consistent with a “shutdown” that was somewhat applied / somewhat honored.

    At any rate, looks like things are in motion again. It’ll be all day unsnarling this one.

    There was a ground hold. Airborne flights continued or were diverted. International flights timed to arrive in AM. International flights overflying US. Only about 20% of departures were on time.

    • #33
  4. RufusRJones Member
    RufusRJones
    @RufusRJones

    lol

     

     

     

    • #34
  5. RufusRJones Member
    RufusRJones
    @RufusRJones

    Just reporting…

     

     

     

     

    • #35
  6. The Cynthonian Inactive
    The Cynthonian
    @TheCynthonian

    Skyler (View Comment):

    Michael Minnott (View Comment):

    It could be penny-pinching if the cause of the database failure was their IT department not being up to snuff. Equipment failure, lack of redundancy, software not updated, lax security, etc. Any number of things could cause it if they cut corners and weren’t updating their IT like they should.

    Eh, maybe. But you’d think accountants would understand that basic equation. They might not understand the art of customer satisfaction, but business school certainly teaches them that running an IT system is important.

    Actually, these days it doesn’t.   It teaches them that quarterly results are the number one priority.  (Note:  this is also heavily incentivized by current shareholder-empowerment laws.). DIE obeisance and virtue signaling are the second.  IT upgrades and improvements, especially in a business other than IT, have to fight for their share of the fungible budget pie.  

    • #36
  7. WI Con Member
    WI Con
    @WICon

    Bunsen (View Comment):

    And now today (1/11), the FAA has grounded all flights due to a “system outage”. Apparently their National Air Mission System (NAS) went belly up. This system notifies planes in the air about emergencies, etc so they can adjust as necessary. Obviously no real explaniation as to what happened but pilots are speculating that the outdated system is reaching its limits.

    A friend of mine is a LUV pilot and he (and the vast majority of his co-workers) do believe it was the system. There has been no talk of sabotage, etc.

    Of course America’s Mayor will be MIA!

    It maddening that in this Era of Trillion Dollar Deficits and no ridiculous expense General budget item gone unfunded, that system like this, the national electrical grid, the Border Wall, hardening the grid from EMP- these are (relatively) modest expenditures and decades gone by without upgrades.

    • #37
  8. Saint Augustine Member
    Saint Augustine
    @SaintAugustine

    Bunsen (View Comment):
    Of course America’s Mayor will be MIA!

    The important thing is that he’s gay. Competence is unimportant. Focus on what really matters in life.

    • #38
  9. RufusRJones Member
    RufusRJones
    @RufusRJones

    Saint Augustine (View Comment):

    Bunsen (View Comment):
    Of course America’s Mayor will be MIA!

    The important thing is that he’s gay. Competence is unimportant. Focus on what really matters in life.

    I bet there are thousands of ordinary people involved in intermodal shipping that could do that job better. 

    • #39
  10. Saint Augustine Member
    Saint Augustine
    @SaintAugustine

    RufusRJones (View Comment):

    Saint Augustine (View Comment):

    Bunsen (View Comment):
    Of course America’s Mayor will be MIA!

    The important thing is that he’s gay. Competence is unimportant. Focus on what really matters in life.

    I bet there are thousands of ordinary people involved in intermodal shipping that could do that job better.

    The job of being gay? That’s his real job, isn’t it?

    • #40
  11. Bunsen Coolidge
    Bunsen
    @Bunsen

    Saint Augustine (View Comment):

    Bunsen (View Comment):
    Of course America’s Mayor will be MIA!

    The important thing is that he’s gay. Competence is unimportant. Focus on what really matters in life.

    Of course, I get lost on the whole “doing your f***ing job!!!!” part.  Shame on me

    • #41
  12. Doug Watt Member
    Doug Watt
    @DougWatt

    I suspect proprietary software unique to Southwest played a part in their latest debacle. Other issues are Baby Boomer pilots are retiring. Alaska Airlines is trying to navigate through a pilot shortage. The other problem is you can de-ice aircraft but once runways freeze over nothing flies in or out.

    When I worked for TWA, there was always the risk that an undersold 747 would still have to fly to Boston from Heathrow in London. That 747 was scheduled to fly to LAX from Boston with around 330 passengers. We flew one trip like that with 30 people on the 747 from London to Boston, but the Boston to LAX flight was sold-out. You hope that commercial cargo will help cut the cost of flying that aircraft.

    • #42
  13. Saint Augustine Member
    Saint Augustine
    @SaintAugustine

    [Oh, wait. Wrong problem for this dumb joke. Never mind.]

    • #43
  14. Hoyacon Member
    Hoyacon
    @Hoyacon

    I used to fly Southwest because they went from DC to fabulous Greene Airport in the garden spot of Warwick, Rhode Island.  The cattle calls that they felt were “innovative” when boarding have earned my enduring enmity.  

    Oh, and has been suggested above, the main issue is that they appointed a bean counter CEO, as opposed to an operations guy.

    • #44
  15. Phil Turmel Inactive
    Phil Turmel
    @PhilTurmel

    David C. Broussard (View Comment):

    Skyler (View Comment):

    OldPhil (View Comment):

    Bryan G. Stephens (View Comment):

    The explanation I saw was that they did have a computer failure, but it was not sabotage. It was outdated and not up to the task.

    I don’t know why you are intent on seeing malevolence when incompetent greed that is perfectly in line with how companies mature after the founder leaves is the explanation.

    The took the eye off of operations and put it on finance. As a former CEO and an Ops guy, I can tell you, companies make poor operational decisions in order to make money. I was let go without cause because I was not playing that game and focusing on operations. I know the drill.

    The explanation from the Pilots sounds perfectly plausible.

    Yep, everything I read was that the bad weather was just what tipped the whole system over the edge. Their systems couldn’t quickly adapt, couldn’t tell them where all their pilots and attendants were at any moment, etc., etc.

    They’ve had bad weather before. They’ve scheduled pilots before. There was nothing really to indicate that the problem was massively worse than ever before. That explanation simply doesn’t fit.

    They also had snafus before due to weather. In the end, the point to point system is harder to plan slack into, but is more amenable to the passenger. There is a reason that most airlines use a Hub and Spoke model, it’s easier and has redundancy built in. With SW, they don’t have that redundancy and when things gotnsideways, their old system that tracked and assigned people by phone wasn’t able to process things fast enough. Technical Debt is a real thing and it usually comes and bites you at the worst time. Continual investment in IT is a requirement of a well run business, but way too many leaders live by the aphorism that the cobblers kids have the worst shoes and skimp on internal systems thinking that the employees will be OK just working harder. When it blows up it can cripple or destroy your business.

    This is key.  They’ve had this problem before.  Last summer (?), after a smaller one of these IT meltdowns, the pilots’ union placed an IT upgrade ahead of pay raises in their negotiating position for the next contract.  Apparently line operations has been begging management for years to allocate funds to upgrade.

    Yes, there are many finance types who do not believe the technical doomsayers, because they are smarter than the nerds.  If they weren’t smarter, they wouldn’t have been placed in charge of the company, you see.

    • #45
  16. David C. Broussard Coolidge
    David C. Broussard
    @Dbroussa

    Hoyacon (View Comment):

    I used to fly Southwest because they went from DC to fabulous Greene Airport in the garden spot of Warwick, Rhode Island. The cattle calls that they felt were “innovative” when boarding have earned my enduring enmity.

    Oh, and has been suggested above, the main issue is that they appointed a bean counter CEO, as opposed to an operations guy.

    I despised the open searing, but my company decided that Southwest was our corporate airline (and my personal choice of AirTran was bought by them anyway).  We tend to purchase Business Select fares to allow for early boarding and easy changes to travel plans. There don’t have change fees, but if you have to change and you bought a 21 day advance fare you have to rebook at same.day prices often and that’s a big hit. When I was flying to DC all the time I had a system with a coworker to almost never have a middle seat taken. We are both large men (I’m much larger) but we would take a few rows in front of the exit rows and then immediately start reading our phones and never made eye contact. It was about 90% effective from DC to Chicago for six months. 

    Eventually I found that I liked to sit in the same seat every time. Second to last row on the shade side of the airplane. I could get an extender and never had to fight for overhead bin space. I have been able to get that seat on every flight with maybe o e or two exceptions (high number of people already aboard).  That calms me when I am traveling, but I am.pretty weird that way. AirTran business class with larger seats, free alcohol l, and premium snacks was still better. 

    • #46
  17. DaveSchmidt Coolidge
    DaveSchmidt
    @DaveSchmidt

    OldPhil (View Comment):

    Bryan G. Stephens (View Comment):

    The explanation I saw was that they did have a computer failure, but it was not sabotage. It was outdated and not up to the task.

    I don’t know why you are intent on seeing malevolence when incompetent greed that is perfectly in line with how companies mature after the founder leaves is the explanation.

    The took the eye off of operations and put it on finance. As a former CEO and an Ops guy, I can tell you, companies make poor operational decisions in order to make money. I was let go without cause because I was not playing that game and focusing on operations. I know the drill.

    The explanation from the Pilots sounds perfectly plausible.

    Yep, everything I read was that the bad weather was just what tipped the whole system over the edge. Their systems couldn’t quickly adapt, couldn’t tell them where all their pilots and attendants were at any moment, etc., etc.

    They didn’t have enough sticky notes and the map on the wall was too small. 

    • #47
  18. DaveSchmidt Coolidge
    DaveSchmidt
    @DaveSchmidt

    DonG (CAGW is a Scam) (View Comment):

    Today’s headline is that all flights in the USA are canceled, because the FAA is having IT troubles. Pete’s on it.

    Pete’s on it?? 

    • #48
  19. RufusRJones Member
    RufusRJones
    @RufusRJones

    Phil Turmel (View Comment):
    Last summer (?), after a smaller one of these IT meltdowns, the pilots’ union placed an IT upgrade ahead of pay raises in their negotiating position for the next contract.  Apparently line operations has been begging management for years to allocate funds to upgrade.

    I heard this as well from a guy that sounded really credible. 

    I worked in bank regulation a million years ago. It seems weird to me that the government doesn’t do any basic surveillance about this type of thing. Maybe they do and they didn’t act on it, or get on record with them that they were taking a big risk. The software was from the 90s.

    • #49
  20. Phil Turmel Inactive
    Phil Turmel
    @PhilTurmel

    David C. Broussard (View Comment):

    Hoyacon (View Comment):

    I used to fly Southwest because they went from DC to fabulous Greene Airport in the garden spot of Warwick, Rhode Island. The cattle calls that they felt were “innovative” when boarding have earned my enduring enmity.

    Oh, and has been suggested above, the main issue is that they appointed a bean counter CEO, as opposed to an operations guy.

    I despised the open searing, but my company decided that Southwest was our corporate airline (and my personal choice of AirTran was bought by them anyway). We tend to purchase Business Select fares to allow for early boarding and easy changes to travel plans. There don’t have change fees, but if you have to change and you bought a 21 day advance fare you have to rebook at same.day prices often and that’s a big hit. When I was flying to DC all the time I had a system with a coworker to almost never have a middle seat taken. We are both large men (I’m much larger) but we would take a few rows in front of the exit rows and then immediately start reading our phones and never made eye contact. It was about 90% effective from DC to Chicago for six months.

    Eventually I found that I liked to sit in the same seat every time. Second to last row on the shade side of the airplane. I could get an extender and never had to fight for overhead bin space. I have been able to get that seat on every flight with maybe o e or two exceptions (high number of people already aboard). That calms me when I am traveling, but I am.pretty weird that way. AirTran business class with larger seats, free alcohol l, and premium snacks was still better.

    I flew Southwest only a couple times, and only because AirTran bought them.  I, too, miss AirTran’s relatively affordable business class.  (They kept Delta honest.)  I despise SouthWest’s cattle car seating, and won’t fly with them just for that reason.

    • #50
  21. Red Herring Coolidge
    Red Herring
    @EHerring

    Weird. Weren’t they the preferred choice when other US airlines went downhill?

    • #51
  22. Phil Turmel Inactive
    Phil Turmel
    @PhilTurmel

    Red Herring (View Comment):

    Weird. Weren’t they the preferred choice when other US airlines went downhill?

    Not for me.

    • #52
  23. Brian Clendinen Inactive
    Brian Clendinen
    @BrianClendinen

    I don’t know Airline scheduling but I due know Manufacturing scheduling. And I do have a good understanding of databases.

    So most likely, this was like most large failures,  multiple things went wrong at the same time.

    You can have scheduling software that does not change but based on how you use it, it can stop working then when you get something that is complex and or your management decides they don’t want margins of safety in your system so you can meet explanations of customers for highly distributive events. It can colasple.

    So you very well could have had schedulers who knew how to set-up the system to make it work. Then you had mid-level managers trying to cut costs to have no margin of safety. Suddenly your scheduling is set-up badly for major weather events. Then if on the fly you can’t go back and set-up your routes correctly but have been jurry rigging them to keep them working with a massive pilot shortage. You suddenly have badly set-up routes that were not designed for the resource constraints in place.

    So they say its an outdated system. It very well could be it only worked under specific conditions and to save cost they removed some of those conditions therefore the system became brittle.

     

    You have to remember this software is a business process. It’s a tool you interact with you get your output aka process done.

    So unless they got hacked.  This sounds like a business process failure not a software failure. There are many software failures that happen because it was coded with a bad process or it was changed to a bad process that had major issues.

     

    Like it might work if you have a dozen or three dozen direct routes. But if you suddenly have 300 and the resources are centrally managed. It can colasple.  If you keep reducing your margin of dafty aka thruput time it take to get things done and how much slack you have to fill up issues. When a variable that was always minor suddenly becomes material. Your scheduling can have huge issues that lead to massive failure.

    • #53
  24. BastiatJunior Member
    BastiatJunior
    @BastiatJunior

    RushBabe49 (View Comment):
    A lot of aerospace has had problems with bean-counters running the show and emphasizing cost-cutting over safety and operational efficiency.

    The 737-Max is the first thing that comes to mind as an example, but there are lots and lots of others.

    • #54
  25. kidCoder Member
    kidCoder
    @kidCoder

    This is the version I heard. Sure there is the management stuff, but at the guts…

    Southwest flies planes. Let’s say from Boston to Baltimore

    Now let’s say that Boston has a shortage of ground crew. The planes don’t fly.

    But there was supposed to be a pilot on that plane.

    According to Southwest Flight Tracking Software, this is what reality was after the flight was scheduled:

    According to reality:

    Well that’s no good. You have pilots in Boston and you think they are in Baltimore. You’ve subtracted the 1 hour flight from their working hours left that day, and you expect them to be able to serve a flight out of Baltimore. And to be unavailable to serve a flight out of Boston.

    Luckily Southwest saw that this would happen. So they devised a system to fix it.

    You call the admin, say “Yo, I’m stuck”, and they adjust you.

    But what happens if ALL of your pilots out of Boston get stuck?

    And what if you only have a couple admins? And your scheduling app needs updates from all the pilots? And while you were doing those calls, the first pilot was scheduled to go to Buffalo?

    And what if the lack of pilots now mean you end up with hundreds of pilots across the network ALSO calling in to have SouthWest Scheduling.

    And they have to undo all those flights that the system think they took?

    And if they don’t, none of those pilots can fly?


    That is how you end up with crew and airplanes in the same place, but unable to legally put that crew on that airplane. So instead of breaking the law in a way that guarantees severe jail time and fines, you don’t fly the plane. And you don’t fly the next plane. And on and on it goes until you go “STOP! EVERYBODY STOP!” and go around marking down where everybody is located, and wait until Friday morning to start all the clocks and have everything go back to normal.

    At least until you can’t get your flights out of Boston.


    One note, the rumor is the plane backup started in Denver where it was so cold ground crews were not servicing aircraft and thus none of that airport’s aircraft left. This cascaded across the system. Arguably hub+spoke would have helped, if Denver was a spoke and not a hub. With Denver being a hub, no amount of armchair CEOing would fix it.

    • #55
  26. BDB Inactive
    BDB
    @BDB

    kidCoder (View Comment):
    This is the version I heard. Sure there is the management stuff, but at the guts…

    Cut to the chase!  What’s the software to generate the graphs?

    • #56
  27. kidCoder Member
    kidCoder
    @kidCoder

    BDB (View Comment):

    kidCoder (View Comment):
    This is the version I heard. Sure there is the management stuff, but at the guts…

    Cut to the chase! What’s the software to generate the graphs?

    https://github.com/terrastruct/d2

    Playground (and one of the charts) at https://play.d2lang.com/?script=ggeflUIRyPdcTvnFJfl5egGZOfklVgrVYG9YKRSkFhXn59VyOSXmlGTm5heloqhT0LVTQMhwAQIAAP__&

    • #57
  28. Skyler Coolidge
    Skyler
    @Skyler

    Phil Turmel (View Comment):

    David C. Broussard (View Comment):

    Hoyacon (View Comment):

    I used to fly Southwest because they went from DC to fabulous Greene Airport in the garden spot of Warwick, Rhode Island. The cattle calls that they felt were “innovative” when boarding have earned my enduring enmity.

    Oh, and has been suggested above, the main issue is that they appointed a bean counter CEO, as opposed to an operations guy.

    I despised the open searing, but my company decided that Southwest was our corporate airline (and my personal choice of AirTran was bought by them anyway). We tend to purchase Business Select fares to allow for early boarding and easy changes to travel plans. There don’t have change fees, but if you have to change and you bought a 21 day advance fare you have to rebook at same.day prices often and that’s a big hit. When I was flying to DC all the time I had a system with a coworker to almost never have a middle seat taken. We are both large men (I’m much larger) but we would take a few rows in front of the exit rows and then immediately start reading our phones and never made eye contact. It was about 90% effective from DC to Chicago for six months.

    Eventually I found that I liked to sit in the same seat every time. Second to last row on the shade side of the airplane. I could get an extender and never had to fight for overhead bin space. I have been able to get that seat on every flight with maybe o e or two exceptions (high number of people already aboard). That calms me when I am traveling, but I am.pretty weird that way. AirTran business class with larger seats, free alcohol l, and premium snacks was still better.

    I flew Southwest only a couple times, and only because AirTran bought them. I, too, miss AirTran’s relatively affordable business class. (They kept Delta honest.) I despise SouthWest’s cattle car seating, and won’t fly with them just for that reason.

    The “cattle car” seating is brilliant.  It gets people motivated to sit down and reduces loading time significantly.  Jets don’t make money sitting still and I’m happy for anything that gets the plane moving. 

    • #58
  29. BDB Inactive
    BDB
    @BDB

    Skyler (View Comment):
    The “cattle car” seating is brilliant.  It gets people motivated to sit down and reduces loading time significantly.  Jets don’t make money sitting still and I’m happy for anything that gets the plane moving. 

    I’m not above throwing an elbow for an exit row window seat.

    • #59
  30. The Cynthonian Inactive
    The Cynthonian
    @TheCynthonian

    Skyler (View Comment):

    Phil Turmel (View Comment):

    David C. Broussard (View Comment):

    Hoyacon (View Comment):

    I used to fly Southwest because they went from DC to fabulous Greene Airport in the garden spot of Warwick, Rhode Island. The cattle calls that they felt were “innovative” when boarding have earned my enduring enmity.

    Oh, and has been suggested above, the main issue is that they appointed a bean counter CEO, as opposed to an operations guy.

    <Snipped for length>

    I flew Southwest only a couple times, and only because AirTran bought them. I, too, miss AirTran’s relatively affordable business class. (They kept Delta honest.) I despise SouthWest’s cattle car seating, and won’t fly with them just for that reason.

    The “cattle car” seating is brilliant. It gets people motivated to sit down and reduces loading time significantly. Jets don’t make money sitting still and I’m happy for anything that gets the plane moving.

    I dislike the “cattle car seating” and because of it, I only fly Southwest if its schedule is the best option and the price  is right.   I don’t perceive it as significantly reducing loading time.  Maybe it reduces some labor cost by eliminating employee time spent shuffling seat assignments so parties can sit together, especially when flights are late or cancelled.  I’ve never understood why the airlines don’t board the passengers from the windows in.  I recall Delta experimenting with it some years ago, but apparently it didn’t prove out.  Maybe because of parties with children and other passengers who need help with the boarding process?

    • #60
Become a member to join the conversation. Or sign in if you're already a member.