On the taxi out 26 February 2007, after starting the right engine, the flight crew of the United Airlines B777 at Heathrow heard a low “growling” noise from below. What they didn’t know was that the unusual sound was of electrical arcing.
The arcing dripped molten metal on insulation blankets, which ignited, spreading the fire and the below-deck space filled with thick smoke.
The pilots got a message for “ELEC AC BUS R” failure, but their fire detection system remained silent. The smoke warning was triggered on the flight data recorder (FDR), which is nice for historical review and analysis, but a smoke/fire alert was never presented in the cockpit.
“Something happened,” said the First Officer. “I just saw my … panel went bonkers.” He was referring to the fact that is Primary Flight Display and Navigation Display had momentarily blanked.
The cockpit voice recorder (CVR) picked up the sound of cooling fans powering down.
The First Officer remarked, “Oh, that is not a good sign. The whole right main [electrical] bus just crashed.”
Thinking they had an electrical failure, the two pilots elected to taxi to a nearby parking stand. The ground handling crew noticed smoke venting from the aircraft. Fire fighters were dispatched; they entered the main equipment center below decks, where they were greeted by thick clouds of black smoke.
They did not detect flames, so the smoke was cleared, which revealed obvious signs of fire damage. The 185 passengers aboard exited via air stairs, and the UK’s Air Accidents Investigation Branch (AAIB) was called to investigate. The AAIB released its report 16 April. It’s a fine piece of work, explaining how an electrical failure led to a spreading fire. Whether its recommendations go far enough may be a matter of opinion. The AAIB’s report reveals that the electrical fire probably started in an electrical equipment box. As the AAIB dryly concluded:
“The overall severity of the damage made it impossible to determine the initiating point of the failure, but the damage was consistent with a high degree of heat generated from multiple arcs and short circuits.”
“P200 power panel with cover installed. The location of the Right Generator Circuit Breaker (RGCB) and Right Bus Tie Breaker (RBTB) is the cover. IDG is the Integrated Drive Generator. Locating the RGCB and the RBTB (and the left side equivalents of same) in the same box may not be a good design practice. A case could be made that these two critical, interactive and fire-prone components should be segregated.”
It could not be conclusively determined if the arcing began in the right generator circuit breaker (RGCB) or the right bus tie breaker (RBTB), both of which were in the same box.
“Fire damage to the P200 power panel with cover removed, showing burnt-out RGCB and RBTB contactors (view looking forward and to the right).”
Fire damage to the P200 power panel with cover removed, showing burnt-out RGCB and RBTB contactors (view looking forward and to the right).
The AAIB explained:
“If the RGCB had failed first and was suffering from internal arcing, then it is quite possible that within 23 seconds the casing became compromised and molten metal dropped down onto the RBTB below. This molten metal would have compromised the RBTB’s casing and initiated short circuits and arcing within it. The GSR [ground series relay] sits directly below the RBTB and it was clear from the GSR damage that it had been compromised by molten material dropping down from the RBTB.”
“Comparison of the Right Generator Circuit Breaker (RGCB) and the Right Bus Tie Breaker (RBTB) contacts with a new contactor held in position. The damage to the Ground Series Relay (GSR), located directly below the RBTB, appeared to have been caused by molten metal dropping down from the contactors above.”
Comparison of the Right Generator Circuit Breaker (RGCB) and the Right Bus Tie Breaker (RBTB) contacts with a new contactor held in position. The damage to the Ground Series Relay (GSR), located directly below the RBTB, appeared to have been caused by molten metal dropping down from the contactors above.
One thing is clear, though, from the AAIB report:
“The internal arcing of the RGCB and RBTB continued unchecked for a long time without activating any protection logic, besides the erroneously [emphasis added] activated ‘Feeder Differential Fault Protection.’ ”
Ignore the acronyms. The important point is that one compromise led to another, the classic definition of a cascading failure.
Wires were burnt. As the AAIB report summarized:
“Some of the wires from the P205 panel were damaged and resulted in some spurious warnings being recorded by the FDR [flight data recorder], but none would have affected the safety of flight. Additional wiring damage in this bundle would have triggered false warnings to the flight crew, including the possibility of a false left engine and right engine fire warning. If this had occurred in flight it would have proved very distracting and confusing to the flight crew.”
As it was, the situation on the ground was confusing enough to the crew and was serious enough to negate any thoughts of flying the aircraft. The damage in the electrical box was estimated to exceed 1,000º C (1,832º F). By way of comparison, copper (Cu) and aluminum (Al) wire melt at 1,080º C (1,976º F) and 450º-560º C (840-1,040ºF) respectively. The fire was searing enough to burn and char insulation, exposing conductor. The molten droplets of metal were hot enough to ignite the insulation blankets.
One could not intentionally construct a more effective accident chain:
- The open base of the P200 panel allowed molten metal droplets from the failed contactors to drop down onto the insulation blankets and ignite them.
- The aircraft’s electrical protection system was not designed to detect and rapidly remove power from a contactor suffering from severe internal arcing and short circuits.
- The contactors had internal design features that probably contributed to the uncontained failures.
Let us briefly consider each link in the accident chain.
First, the electrical box was open on the bottom, which permitted the globs of molten conductor to drop through, igniting the insulation blankets below. The AAIB recommended the installation of a Boeing-designed containment tray beneath the open power panels (to prevent ignition of the insulation blankets). The tray was available for retrofit on all B777s 20 July 2007 – five months after the accident – but has yet to be mandated by the Federal Aviation Administration (FAA) fully 26 months after the accident (which, as the AAIB documented, capped a series of previous similar arcing failures). As the AAIB pointedly noted: “The AAIB considers the time elapsed from the issuance of the Service Bulletin in July 2007 to the as-yet unpublished AD [airworthiness directive] to be unacceptable …”
The AAIB could have gone further, recommending that the FAA evaluate other transport category aircraft to see of they were vulnerable to similar dripping blobs of arced conductor and, if so, mandating similar protective trays.
Regarding the insulation blankets, they were fiberglass batting covered with a polyethelyne terephthalate (Mylar) film. Recall that after the Swissair crash in 1998 from an in-flight fire that all metalized Mylar insulation material was ordered by the FAA to be removed from Douglas-built jets and replaced with more fire-resistant blankets. This was only a partial solution to the general problem of flammable materials, as it was known at the time that other materials installed in the fleet did not pass the FAA’s test for fire resistance, either, which included Mylar.
“Estimated temperatures of insulation blankets directly under the P200 power panel, looking down from the panel.”
Estimated temperatures of insulation blankets directly under the P200 power panel, looking down from the panel.
As the AAIB observed:
“In 2005 the aircraft manufacturer introduced a new type of insulation blanket which used the same fiberglass batting but had a new covering film that was more resistant to radiant heat. This insulation was developed in order to pass a more stringent fire test involving a 2,000º F (1,039º C) propane flame whilst exposed to radiant heat. This type of insulation was not fitted to [the accident aircraft] and there was no requirement for retrofit.”
The AAIB could have strongly urged the FAA to get on with such insulation blanket retrofit but was strangely silent on the matter.
“Cotton swab test of Mylar insulation blanket. As the AAIB report noted: “The longest burn time of any sample was just over 7 minutes … The burn areas from the cotton swab test were small compared to the extensive burn damage observed in the actual event, but in the actual event the insulation blankets were exposed to multiple molten metal droplets.” The test, it seems, is designed to yield “good news” results that will not necessarily be replicated in real world conditions.”
Cotton swab test of Mylar insulation blanket. As the AAIB report noted: “The longest burn time of any sample was just over 7 minutes … The burn areas from the cotton swab test were small compared to the extensive burn damage observed in the actual event, but in the actual event the insulation blankets were exposed to multiple molten metal droplets.” The test, it seems, is designed to yield “good news” results that will not necessarily be replicated in real world conditions.
Second, the arcing continued for a long time without activating any protection logic. On this score, the AAIB report language is one of frustration:
“If the failure occurred inside the BTB [bus tie breaker] then the bus would be isolated by opening and locking out the APB [auxiliary power breaker] and opposite BTB. This would isolate the fault. However, the aircraft manufacturer argued that implementing such a change would involve a costly redesign, re-qualification and re-certification of the power panels and the system.”
Boeing told the AAIB that “containment tray modification should be sufficient.”
“In order to prevent molten metal from a failed component dropping down on insulation blankets and igniting them, a containment tray modification has been proposed. Boeing recommended installation within five years but this is advisory, not mandatort. The FAA has yet to require this modification, which installs a 1.6 mm (0.06 inch) thick aluminum tray at the open base of the P100, P200 and P300 power panels.”
In order to prevent molten metal from a failed component dropping down on insulation blankets and igniting them, a containment tray modification has been proposed. Boeing recommended installation within five years but this is advisory, not mandatort. The FAA has yet to require this modification, which installs a 1.6 mm (0.06 inch) thick aluminum tray at the open base of the P100, P200 and P300 power panels.
The AAIB recommended that Boeing “should consider” implementing differential current fault protection, but it seems that the manufacturer has already considered and rejected this idea. It might have been better to enjoin the FAA to study the improved circuit protection for all aircraft and to mandate changes to certification criteria, applicable to all future designs.
At least action is being taken to alert crews to smoke in the equipment bay. The AAIB recommended such warning be mandated b y the FAA “at the earliest opportunity, with a software update that will generate a caution message to alert flight crew of the presence of smoke in the Main Equipment Centre.”
Let’s put this in context: here we are, two years after this event, crews still don’t have such a warning and, given the length of time the FAA takes to action (Notice of Proposed Rulemaking. Implementation time, etc.), it will be at least another two years to accomplish fleet wide installation of such a warning.
The third link in the accident chain was the design of the circuit breaker contactors. The accident prompted numerous changes to the design of the circuit breakers to make them safer in performing their function – which is not to arc but to prevent arcing. Modifications were planned for the end of 2008 but there is no requirement to retrofit the new breakers onto existing aircraft.
The AAIB recommended that regulators order that the old style contactors be replaced by ones of more recent design “to reduce the risk of a contactor breakdown that results in uncontained hot debris.”
It’s pretty evident from the AAIB report that in-service deterioration of components (electrical and blankets) have been highlighted by this accident and a review of prior arcing events. The report is a real wake-up call that “on condition” (i.e., hasn’t failed YET) is a poor method for monitoring against impending or imminently potential catastrophic failure. One example the AAIB presents of contact wear suggests that 45,700 hours and 14,500 flight cycles may be an excessive in service “awaiting failure” period. It is quite clear that higher time contactors are degraded (and it would appear that they do so exponentially as erosive spatter wear “sets in”).
Example of a contactor on a right bus tie breaker with high main contact erosion after 47,000 flying hours and 14,500 flight cycles.
Example of normal contact wear on an auxiliary power breaker that has completed 25,000 flying hours and 22,000 flight cycles.
It should also be noted that once an airframe is designed, it undergoes pressure cycles to the point of failure to demonstrate inherent integrity equal to or greater than the airplane’s planned service life. Perhaps comparable testing should be done with high current carrying electrical systems to uncover the typical failure modes and the weak points in the assemblies. Of interest here are the fatigue failures and di-electric breakdowns that occur due to thermal cycling. Operational experience shows that material changes have been necessitated. Each such failure has created an unwarranted hazard to life and limb. It’s time such full-up, lifetime electrical system testing be accomplished before, not after, passengers are aboard as the unwitting (and doubtless unwilling) participants.
There you have it, the sum of the AAIB’s findings and recommendations. They address same pretty basic, rather glaring deficiencies. One has to ask, how do such obvious shortcomings – like the absence of a protective shield on the bottom of the power panel – get through the design certification process. And how did the FAA miss all this given that there were at least four events previous to this one where electrical malfunction resulted in major damage to in-service B777s. There is such a thing called pro-active safety, but the FAA’s passiveness is anything but.