Plane engines are extremely highly optimised, and the design margins are basically at the limit of what regulations allow, which is IMHO why the tiniest of errors can have huge consequences. However, in this case it sounds like the manufacturing process was such that it invited misalignment:

Finally, working from the inside through the inner hub counter bore, the stub pipe counter bore would be drilled to a specified depth to accommodate the filter. (This was the bore that was later found to be offset by half a millimeter.)

Why not machine the stub pipes to their desired dimensions, including the drilling, and then install them? That way, no in-situ drilling, with that accompanying risk of misalignment, would even be necessary. Even if they decided such drilling was unavoidable, a drill with a pilot (no pun intended) would've kept the holes as concentric as needed. (Disclaimer: Not a professional machinist; but have done some lathe and mill work.)

The temperature inside this buffer space was likely between 365 and 375˚C, well above the 280-degree auto-ignition temperature of the engine oil, so the spray immediately ignited.

It's worth noting that due to fire risk, most planes use non-flammable hydraulic fluid, although it's not without its own dangers, so perhaps non-flammable engine oil was considered but ultimately decided against: https://en.wikipedia.org/wiki/Skydrol

I flew on this exact same plane, VH-OQA, on November 4, 2019, from Singapore to Melbourne.

While I was sitting at the gate, I remembered that line from Rain Main about "Qantas never crashed," and since that flight would be my first time flying on an A380, I wondered if any had crashed or had any other incidents. I looked up "a380 crash" on my phone, and what do you know? The first hit was a Wikipedia article about a Qantas A380 uncontained engine failure, which had occurred four minutes after takeoff... from Changi! Even wilder was the fact that it had happened 9 years earlier. When I looked up the plane's registration number on Flightradar24, I realized that was the same plane I was flying to Melbourne on.

• gregjw

  today at 2:35 AM

  yikes, im sure that didnt freak you out at all.

  • Insanity

    today at 4:33 AM

    I'd be insanely stressed about this. While at the same time realizing that logically it has flown for _years_ without incident afterwards.

Turbine disk failure is one thing that can down a plane, and you can only reduce the probability so much.

Fan blades and turbine blades can be contained by containment cases if they break off, but turbine disks can't. When a disk breaks, it's an uncontained engine failure. A 20 kg disk fragment flying at Mach 1 goes through everything.

Airframes are have a 30-degree fragment spread cone around every disk stage. Airframes are designed so that no single fragment can destroy all redundant systems at once in this zone (flight controls, fuel lines, and hydraulic systems). Jet fuel cannot be stored in the wings in this zone.

• chasil

  yesterday at 9:04 PM

  I admit that I glossed over the section on the machining of the oil pipe, but the facility appears to bear most of the blame.

  These were pulled and scrapped from the whole A380 fleet.

  "Investigators also criticized the culture within the Hucknall facility that manufactured the HP/IP bearing hubs, identifying signs of complacency and widespread procedural non-compliance. A paperwork review showed that the required signatures were missing from 131 out of 138 retrospective concessions issued between 2009 and 2011, and a large number of minor non-conformances had not been properly handled through the normal chain of command."

  • jacquesm

    yesterday at 9:35 PM

    That's an oversight failure as much as a failure of that plant.

    • NetMageSCW

      yesterday at 11:39 PM

      The oversight was at the plant.

      • jacquesm

        today at 1:34 AM

        Yes, but there are levels of oversight and this kind of thing should not be possible to hide. Someone on the official side of this messed up as well.

• mcapodici

  today at 1:12 AM

  It is worse than even that:

  > Within the space of four seconds, the energy from the annulus gas flow accelerated the IP turbine past its critical speed, until centrifugal forces exceeded the ultimate strength of the nickel alloy disk. The red-hot, wildly spinning disk instantly fractured into several sections, which rocketed outward in multiple directions at incomprehensible speed.

  • mr_toad

    today at 3:22 AM

    “It goes without saying that if any of the turbine fragments had entered the passenger cabin, there would have been injuries, if not fatalities, even if the plane later landed safely.”

    They got lucky.

• usefulcat

  yesterday at 10:00 PM

  > Jet fuel cannot be stored in the wings in this zone.

  TFA describes a fuel leak resulting from a piece of the turbine disk passing through a fuel tank located within the wing:

  "When the fragments of the IP turbine disk passed through the wing and belly of the A380, they caused considerable secondary damage along the way, not only to the №2 engine but also to the left wing fuel tank, which sprang a leak"

  It also includes a photo of the inside of the damaged wing tank.

  Was the requirement you describe added after this incident?

  • ahartmetz

    yesterday at 10:15 PM

    Seems close to impossible for existing designs. The wings are commonly almost entirely full of fuel at max fuel, and I doubt that taking the capacity hit is an option even in newly designed planes.

• amelius

  yesterday at 9:58 PM

  But it seems to me that the whole failure could have been prevented with a bunch of temperature sensors spread throughout the engine, and a good anomaly detection system.

  • Farmadupe

    today at 2:18 AM

    Don't know why you're getting faded out, because having worked on exactly this engine, software mitigations were put in place. That's exactly your `anomaly detection system`

    It was a belt-and-braces fix, because the primary fix was of course to manufacture the engines correctly in the first place.

    The "lots of temperature sensors" thing is actually easier said than done; jet engines already have lots of temperature sensors, and they will turn the engine off if a huge overtemperature is detected, but it's not a free win to add more of them. It increases the risk of a faulty sensor shutting down a perfectly good engine in flight, which from a safety-critical perspective is a borderline disaster condition all by itself.

    (technically you usually cross-correlate temperature sensor readings with pressure sensor readings to prevent sensor faults turning engines off, but still, extra sensors isn't always a free win.)

• polishdude20

  yesterday at 8:43 PM

  What's a turbine disc?

  • gumby

    yesterday at 8:44 PM

    It’s explained in the article

Can anyone explain why on earth you would counter bore a pipe at all, let alone after installation? I read the article but couldn’t understand the explanation. It seems like a totally ludicrous way to make a pipe.

One of my favourite memories of flying Qantas when I was younger was the inflight entertainment.

If memory serves, they were celebrating some sort of milestone like "60 years without a fatal accident" or similar. Quite an impressive feat!

I'm unclear if it was directly related. But the Qantas inflight entertainment system on our flight from Sydney to Los Angeles meanwhile contained about 12 seasons of the National Geographic series "Air Crash Investigation" (Mayday: Air Disasters for Americans)

I will admit that show made an 18 hour slog of a flight significantly more enjoyable

• Insanity

  today at 4:35 AM

  Lol, I quite like the show and considered watching it. But then thought other passengers on the plane might see it and become a bit unsettled by it.

  So last few times I opted for "To Catch A Smuggler". Still flight related but hopefully less unsettling.

• AdamJacobMuller

  today at 1:12 AM

  Watching Mayday on flights is one of my favorite things to do for some odd reason.

  • annzabelle

    today at 4:31 AM

    There's a podcast (that's been on "hiatus" for 6 years) that I used to listen to that would do an air crash segment. The show was called Hello Internet, and I would always try to queue up the air crash episodes for any flights I was on.

    One of the hosts recently started a new podcast called Runway 23 that's mostly about air crashes, and his cohost is actually a Qantas pilot.

• alex1138

  today at 2:14 AM

  That's hilarious. They don't think about "hey, maybe not show this, we might have nervous passengers"?

The whole incident was because of 0.2mm offset in a machined part! And the company that made the part ignored the quality compliance process was not fired. I wonder why - did Airbus think it’s their failure not to verify that the subcontractor adhered to procedures?

• quink

  today at 2:29 AM

  This feels more like engine manufacturer QC rather than anything Airbus.

  Because also, from Wikipedia: On 22 June 2011, Qantas announced that it had agreed to a compensation of AUD$95 million (£62 million/US$100 million) from Rolls-Royce. - skipping Airbus entirely.

I once heard someone describe Admiral Cloudberg's air crash investigation blog as "the best thing on the Internet" and I still think that might be true. She is a great researcher and a superb writer, which stands out all the more these days in these fallen times of load-bearing AI slop. Her blog is beloved of both aviation geeks and, for obvious reasons, SREs (and other engineers).

She recently posted what's basically her magnum opus, on the Potomac river midair collision.[0] It's the length of a short book. I'm working my way through it; I'm taking a flight today, and I'll probably read it on the plane tonight.

This article, on how an A380 with 469 souls aboard almost crashed, but didn't, is one of her best. I like it because it's a happy ending, and a story of the combined effects of good engineering design (the astonishing degree of redundancy built into the A380 proved just enough to prevent catastrophe), and human troubleshooting under pressure in the cockpit by some smart people with vast experience who remained calm throughout. (Another favorite article of her, on the Air France 447 crash, tells the exact opposite story: poor systems design combined with pilots basically losing the most basic ability to aviate in the heat of the moment. That one is a much darker read.[1])

[0] https://admiralcloudberg.medium.com/reaping-the-whirlwind-in... [1] https://admiralcloudberg.medium.com/the-long-way-down-the-cr...

  > The detailed software allowed them to enter various parameters including the weather, runway condition, and any systems failures, and then calculate whether it was possible to land. But when everything had been entered, the program spit out an unhelpful answer: “no result.”

 > When calculating the landing distance, the software applied a generic “operational coefficient” to account conservatively for variations in pilot techniques that could result in less efficient deceleration. The problem, as investigators would later discover, was that the software applied the coefficient again whenever another system failure was added. With so many system failures on the aircraft, the coefficient was applied a total of 9 times, resulting in a calculated landing distance considerably greater than the length of the available runway. However, Check Captain Evans was able to fix the problem by manually entering their actual landing weight, overriding the program’s assumption of a maximum landing weight. By specifying a landing weight in excess of the maximum, the system logic changed to apply the operational coefficient only once — for unrelated and obscure reasons — and lo and behold, when he ran the numbers this time, the computer said they could just barely land on any of the 4,000-meter runways at Singapore Changi Airport, with only 100 meters to spare.

disc fragments didn't kill anyone just by accident. Have they not considered not seating anyone in the plane of the discs...

This remarkable event, along with the an accessible history of the evolution of precision manufacturing is covered in Simon Winchester's excellent book The Perfectionists[0]. The audiobook is narrated by the author, which is quite enjoyable as well.

[0]https://www.harpercollins.com/products/the-perfectionists-si...

Retroactive, but one thing that struck me since they were overweight for landing and their fuel dump was inoperative, why not turn on the leveling pump and pump fuel out the hole in the left tank anyway? It would have lowered weight and improved balance.

• vient

  today at 12:18 AM

  I got an impression from the text that pilots were not sure the fuel levelling line does not have a leak itself. Maybe they were afraid the line can leak fuel into fuselage somehow?

  • sleepy_keita

    today at 1:30 AM

    They got a warning that the fuel transfer system was damaged, so they decided not to use it, I think.

Unbelievably comprehensive coverage from the Aviation Herald on this incident below, the owner of which was sued recently over a report to the point he sadly lost his home of over 40 years:

https://avherald.com/h?article=43309c6d&opt=0

• chinathrow

  yesterday at 8:26 PM

  Context to the lawsuit brought up by Go2Sky: https://avherald.com/h?article=53c8958a

Also this documentary is very good: https://m.youtube.com/watch?v=nZS-mkygGWI

I wonder why the engineers didn't specify a tolerance for thickness of the oil stub pipe?

• bahmboo

  today at 5:24 AM

  They did. Through a series of complex interactions it was missed. It’s kind of the whole point of the retrospective in the article.

• okey

  today at 1:13 AM

  per the article, because the stub pipe counter bore had to be drilled after that end of the pipe was welded in place, so directly measuring pipe wall thickness wasn't practical

• NetMageSCW

  yesterday at 11:41 PM

  Difficulty in measuring since it was machined after installation?

"Even the airplane itself ultimately survived: after a marathon repair that lasted 535 days and cost $139 million, the A380 Nancy-Bird Walton triumphantly returned to the skies in 2012."

This quote amazes me. My understanding is that Airbus, Qantas, and probably Rolls Royce were willing to spend this much because it meant that the incident was not a "hull loss" and therefore not as bad a hit to reputation and safety statistics. Does anyone know if that's true?

• ygouzerh

  today at 3:17 AM

  One thing that might be taken into account is the delay between ordering a plane, and the delivery (can take up to 10 years), so 1.5 years and 139$ million is worth investing into, to keep revenue flowing earlier.

• ahartmetz

  yesterday at 10:24 PM

  A new A380 is more expensive (rule of thumb half of official list price, so about $220 million), and airlines were not talking about retiring A380s yet at the time. So it even made economic sense.

  • stephen_g

    today at 1:05 AM

    The plane has since had a cabin upgrade and is still expected to operate at least until 2028. It’s operating SIN-LHR right now as I write this, currently flying over Azerbaijan.

• shawabawa3

  yesterday at 10:16 PM

  They paid an estimated $300M per aircraft, so even a $139M repair bill is worth it

Title is: A Matter of Millimeters: The story of Qantas flight 32 (2023)

Some discussion:

2023 https://news.ycombinator.com/item?id=38586773

https://news.ycombinator.com/item?id=49511213

Qantas is a troubled airline. Once a Qantas aircraft technician even told me that Singapore Airlines have a far superior maintenance schedule for their aircraft.

This person claims that he tells his family to avoid flying on Qantas ( but flies on it himself since he gets free flights)

These days Han airlines ( SIA, Cathay,China Eastern) and Middle Eastern airlines (Emirates,Qatar) are far better than Western Airlines ( American, British, Lufthansa, KLM,Qantas).

• tomhow

  yesterday at 8:59 PM

  Please don’t post off-topic hearsay like this here. This article isn’t about maintenance, it’s about a manufacturing fault and the incredible work by the pilots to avert a colossal aviation disaster. Qantas continues to have an unblemished record of safety. Running airlines cost-effectively is a continually growing challenge. There is always a place for scrutiny of airlines’ maintenance and safety practices, but introducing it out of context and based entirely on an unverifiable comment from an unidentified individual is not a helpful contribution to this discussion.

• HDBaseT

  yesterday at 10:32 PM

  Despite this, Qantas is consistently rated one of the top 3 safest airlines in the world every year.

• ginko

  yesterday at 9:23 PM

  Ah yeah, China Eastern..

  https://en.wikipedia.org/wiki/China_Eastern_Airlines_Flight_...

  Meanwhile Lufthansa had their last fatal crash in 1993. For British it was in 1976.

Catastrophic failure? Check

Plane landed without casualties? Check

Plane design so good it helped avoid casualties? Check

Hum... Ah! Not an american Boeing but a french Airbus. Of course.

The article mentions this story should make everyone a little less afraid of flying.

Sure, as long as its not on a Boeing plane.

• jpgvm

  today at 12:35 AM

  The plane was a significant factor but you really shouldn't look past the calibre of expertise in the cabin on this particular flight.

  > 5 pilots were in the cockpit of this flight. In addition to the normal crew of pilot-in-command and co-pilot, there was an experienced relief pilot and two additional check captains; one was being trained as a check captain (CC) and the supervising CC, who was training the CC.

  Captain Richard Champion de Crespigny had 35 years of flying experience at the time of the incident.

  That is an insane amount of flight experience in the cockpit and all of the reports state this was a very significant factor. i.e by allowing distribution of the task load, plenty of time for one of the pilots to compute landing calculations manually etc.

  THe A380 is a flying tank no doubt, but it wasn't the sole factor here - I think most people would agree this one was a shared victory.

• sam1714

  yesterday at 9:34 PM

  Like Boeing, there was organizational dysfunction leading to a direct safety issue.

  During the development of the A380, the German and French sides were using different versions of the CATIA PLM software (i.e. CAD), which prevented people from looking at a single unified model of the plane. One concrete problem it caused was major problems with the wiring harnesses, which led to a 2 year delay in the program, and 3 early A380's which couldn't be sold. (Airbus kept one for test, the other two got sent to museums)

  That manifested in this accident because the wing wiring was routed through the engine disk burst axes (a hazard which is explicitly accounted for) with insufficient redundancy. That created the dangerous situation where they couldn't turn off the engine after landing.

  • jacquesm

    yesterday at 9:36 PM

    > That created the dangerous situation where they couldn't turn off the engine after landing.

    Which is vastly preferable compared to all of the dangerous situations just prior to that landing.

    • sam1714

      yesterday at 9:56 PM

      The vastly preferable situation was to have redundant wiring that properly complied with regulations, which was retrofitted in an airworthiness directive (safety recall) after the accident.

      They also required engine software that detected the sudden increase in speed associated with a shaft break and automatically cut fuel.

• renox

  yesterday at 9:55 PM

  Read about the AF447 Rio-Paris crash. Input stick averaging?? That's a huge design mistake which contributed to the crash and AFAIK is still here..

  • echoangle

    yesterday at 11:05 PM

    What’s the alternative? I think averaging with a dual input alert is a pretty good solution. Otherwise you need to find a solution to transition from one stick to the other. You can’t just switch sticks with the press of a button because the input could jump from elevator up directly to elevator down for example.

    • 4fterd4rk

      yesterday at 11:39 PM

      You give them direct feedback and eliminate the need for this completely

• jryle70

  yesterday at 9:56 PM

  An aviation thread with two utterly ignorant posts - [0]. Is that the aviation knowledge level of HN?

  [0] - this one and https://news.ycombinator.com/item?id=49541946

• ginko

  yesterday at 9:33 PM

  Airbus is French, German and British. Not just French.

  • bluebarbet

    yesterday at 11:24 PM

    The state-held stakes are mainly French, German and (less) Spanish. The UK taxpayer currently owns 0% of Airbus, though the UK gets the business of building wings and landing gear.

• seizethecheese

  yesterday at 9:49 PM

  I thought this was a strange insertion of geopolitics into the conversation so I checked out this account’s comment history. It’s good for others to see that some commenters here have a serious axe to grind and are not really “hackers” as their primary interest here. The most recent comments in order:

  > [something about london taxi drivers, admittedly unrelated]

  > I totally get you and in theory you are right. And yet, in the USA you have gerrymandering and apparently _wanted_ Trump. Twice. Something has to be very wrong somewhere.

  > I would suggest "lobbying" is not the correct word to describe all the corruption going on in the current USA administration cesspool.

  >I recently bought a BambuLab 3D printer. Designed and made in China. Right from the unpacking to first print the experience was top Apple-style quality. Indeed I was agreeably surprised.

  > A global non USA success? Plenty. You might want to remove your pink USA glasses and look around. Spotify good enough for you?

  > I am Canadian. Never, ever would i knowingly buy any dairy products from the USA , at any price. We have cows too, and farmers.

  > Its not that it hates its customers , as much as it it a USA-based company that is run under the rules made by the administration. Not that much different from running Red Star OS from North Korea, actually.

  > [Finally something unrelated]