Boeing nears end to 787 fire investigation


In the 15 days since ZA002, Boeing’s second of six 787 flight test aircraft, suffered a fire in its aft electrical equipment bay, forcing a fleet-wide halt in certification testing, the airframer is days, if not hours, away from releasing its findings of its investigation and disclosing the impact to the aircraft’s first delivery, say company and industry sources.

An additional delay to the 787′s entry into service with All Nippon Airways is now a virtual certainty, the length of that delay, however, is yet unknown.

While some analysts have suggested the 787′s first delivery could slip to 2012, an additional delay of more than nine months, Boeing’s previous six delays have never shifted the schedule more than six months at a time. A six month slide beyond today’s February 2011 plan would place handover to ANA around August of 2011, more than three years after its original target.

Equally important in establishing the root cause of the fire, reported to be foreign object debris (FOD) that caused a short in the panel, is ensuring primary electrical system redundancy remains intact if such an incident were to reoccur. One program source indicates FOD clogging an air duct that cools the P100 panel may be a culprit.

The Seattle Times, citing a source close to the program, reported the 787 fleet, including 23 production aircraft in Everett, have been searched for FOD.

Though the fire, which happened while on approach to Laredo, Texas, and its root cause, revealed an Achillies heel in the 787′s electrical system that must be resolved before the Dreamliner can enter service.

In an internal message to program employees last week, Scott Fancher, 787 program manager and general manager said “we have made good progress in replicating the effects in our integration labs”.

While the specific “effects” have not been disclosed, under normal operation, a drop out of the P100 panel, feeding electricity from the left engine’s twin variable frequency starter generators (VSFG) to aircraft systems, should have compensated by prioritizing the flow of electricity from the right VSFGs into the healthy P200 panel for distribution. Instead, the fire caused 787 to perceive it had lost electrical power completely, causing the Hamilton Sundstrand-built ram air turbine (RAT) to deploy.

The RAT supplies just 10 kVA of electricity, a fraction of the up to 1.45 megawatts of power generated by the aircraft’s primary systems, including the APS-5000 auxiliary power unit (APU). Sources familiar with the incident say the APU, which supplies power to the P150 power distribution panel, was not running at the time of the fire.

The result, was the loss of four of five, heads down displays (HDD) and the twin heads up displays (HUD), as well as autothrottle control and a “cascading” series of failures.

However, Boeing concluded that with its twin Trent 1000 engines still running, ZA002 was “in a configuration that could have been sustained for the time required to return to an airport suitable for landing from any point in a typical 787 mission profile,” a defense of the 2008 Federal Aviation Administration’s special condition imposed on the aircraft’s electrical system, as well as its sought-after extended twin engine operations (ETOPS) certification.

Since their November 9 grounding, Boeing has received permission from the FAA to relocate three 787s. ZA001 and ZA005 returned to Boeing Field from remote testing in South Dakota and California, respectively, and ZA004 was ferried to the company’s Everett facility for maintenance.

Fancher added “re-positioning these airplanes back to Seattle will better prepare us for any modification that are needed as a result of this event.”

The extent of those modifications is another unknown and covers a wide spectrum of possibilities from a limited software modification all the way to complete redesign of the more-electric systems architecture of the 787.

At its inception, the Boeing used the 787 to push the outer envelope aerospace manufacturing, materials and systems, the three attributes that define an aircraft’s development.

The airframer had encountered great pain with the 787′s globally distributed supply chain in 2007 and 2008, a composite structural flaw in the aircraft’s side-of-body in 2009, and now in 2010 a potentially significant change to the aircraft’s electrical system.

Late discovery of design changes following treacherous incidents is not new to Boeing commercial development programs. 

Nearly 28 years to the day earlier, the third of five 757 flight test aircraft suffered severe engine damage during a natural ice build up test on its Rolls-Royce RB211 engines, 

An FAA representative onboard the aircraft, which was also being co-piloted by an FAA pilot, said at the time of the incident remarked: “We almost lost one.”

The incident prompted a late design change engine that was not validated by the FAA until nearly the last minutes before the type was handed over to Eastern Airlines in late 1982, followed by its January 1983 entry into service. 

14 Responses to Boeing nears end to 787 fire investigation

  1. Robin Phillips November 24, 2010 at 1:52 pm #

    A nice article on the Boeing 787 fire, but I find it surprising you don’t mention the campaign Boeing has been waging over the last couple of days(fairly successfully…) to have the pictures of the burnt panel removed from the internet.

    The pictures, whilst they were available, certainly helped bring home the message of how serious the problem was, even before the follow on effects started.

    http://blogs.crikey.com.au/planetalking/2010/11/22/boeing-versus-bloggers

    It is a shame that Boeing (and Rolls Royce on the A380 incident) are so secretive. I don’t think it is a secret that building modern airliners is a very difficult undertaking and just covering up the fact that they’ve made mistakes just makes them look unbelievable. The danger is that they not only cover it up externally but also start to believe the “play it down” message internally – which will get in the way of solving the problem.

  2. CM November 24, 2010 at 3:13 pm #

    @Robin Phillips – Let me understand your post.

    1. You believe believe there’s been an “external cover-up”, which implies more than simply Boeing and Airbus are not releasing everything they know publicly?

    2. You believe the people working these problems inside Boeing, Airbus, Rolls Royce, etc (hundreds of engineers and experts with integrity, professional pride, and families who will someday fly on these aircraft) would just look the other way while some kind of menace to society is sold to the public as safe?

    3. Even if you believe both of the above are true, you must also believe the certifying agencies will also look the other way while the OEMs “drink their own koolaid” and deliver/return unsafe products to the public?

    Honestly people! Do you even think about the things you write? The cluelessness about how the aviation industry works is truly mind-boggling! The belief that any company whose reputation is staked to the safety of their products would sell their soul and sacrifice the reputation of their company in order to save a few weeks (or even months) of schedule impact to a 6+ year program is the height of naivety.

    Jon’s post says “the airframer is days, if not hours, away from releasing its findings of its investigation and disclosing the impact to the aircraft’s first delivery”. Unless you’re disputing this claim, why on earth would you post the comment you did?

    I’m truly baffled.

    CM

  3. Wanabee Engineer November 24, 2010 at 3:25 pm #

    By any chance was this FOD located in the liquid cooling loop? I thought P100 panels were liquid cooled. If it is the liquid cooling loop, then the loop filter is likely the cause. HS has that responsibility also…

  4. reader November 24, 2010 at 4:11 pm #

    “One program source indicates FOD clogging an air duct that cools the P100 panel may be a culprit.” What part of air duct makes you think water cooling!?!

  5. Uwe November 24, 2010 at 4:45 pm #

    the information that the P100 panel or some part of it
    is said to be watercooled?

    IMHO Boeing got a real fright from this one.
    It will be the first really unexpected delay
    in sight. ( or was someone gnawing his/her
    knuckles all the time for exactly this scenario
    not to happen )

  6. WingBender November 24, 2010 at 6:03 pm #

    Thanks CM. Excellent rebuttal.

    “Conspiracy theories were invented so stupid people can pretend they’re smart.”

  7. Link Building November 24, 2010 at 8:47 pm #

    You make blogging look like a walk in the park! I’ve been trying to blog daily but I just cant find writing material.. you’re an inspiration to me and i’m sure many others!

  8. Daniel Tsang November 24, 2010 at 9:08 pm #

    The most noteworthy point is, in Boeing’s statement, it said there is only a “minor” design change to the software and P100 panel design.

    However, despite the fact that I am a big supporter of Boeing 787 Dreamliner, I think it has to be cautious in releasing new schedules and build more margins into the schedules.

    Regrettably, I think the cumulative impact of the previous (6th) delay and the fresh 7th delay, means that the Entry Into Service (EIS) of 787-9 derivative as well as the 787 production rampup to 10/month by end-2013 will definitely be delayed, thereby hurting BA’s financial performance.

    http://www.aspireaviation.com/2010/11/22/787-potential-seventh-delay/

  9. Aero Ninja November 25, 2010 at 3:00 am #

    Right. First I think CM and Wingbender ought to read what Robin PhilIips actually wrote. Nothing about conspiracies.
    He bemoans the fact that Big Brother Boeing ensured that the pictures of the damaged panel were removed from the internet and that the news media and bloggers all meekly complied. Basically a closing the barn door after the horses have all fled situation anyway!

    Or does Boeing want to do a search of everybody’s hard drives, the world over, to ensure that none of these pictures still exist?!

    You are the ones to use the word conspiracy. Robin Phillips expressed the opinion that covering things up leads to a lack of credibility (something I agree with and something that even the most ardent Boeing fans are slowly having to confront) and can lead to a negative internal company culture (also not unheard of, or should we all forget the empty shell that was presented at the grand and glorious 7-08-07 rollout ceremony?).
    I think you two should stop being so patronizing.

    Lastly, I don’t think there is a conspiracy here with the certifying authourites but if, let’s face it, there is a reason why we need them. I believe the Boeing engineers, produciton workers etc. all take pride in their work and want a safe product but we still need someone to watch over Boeing (and Airbus, Bombardier, Embraer, etc) as a whole. And I don’t think what Robin Phillips wrote was so wrong or bad.

  10. Robin November 25, 2010 at 6:45 am #

    CM:

    I think you miss-understood what I meant by “external”. This was supposed to mean that Boeing is trying to suppress information in the public domain (ie. “external” to the company). I don’t think this is good public relations, on the contrary, it makes what they do say unbelievable.

    I have no problem with the fact that they find problems during certification and testing. That is the whole point of doing the testing, to find these problems before the plane is used commercially. It would be extremely surprising if anyone managed to build something as complex as a modern airliner without finding something that needs improving when you test it.

    As to your point(2)- no I don’t believe people deliberately cover it up. The danger is that the same message you give out publicly (ie. “it wasn’t that serious, we’ll make a few minor changes and then don’t need to worry about it any more”) gets absorbed internally as well. Then rather than saying “this is serious, we need to do a lot of work to fix the underlying cause”, you end up deciding “lets just patch this over with a quick fix and move on”. This has nothing to do with integrity but simple human nature of avoiding a difficult decision that will cause lots of extra work when you are snowed under with work anyway.

    If you don’t think this is realistic then just look at what seems to have happened at Rolls Royce – they seem to have know about the problem with the leaking oil in the engines previously. They clearly made the decision to “cover it up” by not telling their “external” customer (Qantas) and as a result didn’t treat it seriously enough “internally” (they seem to have decided to gradually apply the fix over a longer period of time). This was clearly the wrong decision and could easily have killed >400 people. It wasn’t because they had no integrity, it was simply the wrong decision but one that might not have been made if they had been more open “externally” since they would have faced much more questioning from their customer about how serious the problem really was.

    That discussion is what is missing here with the 787 as well.

  11. Link Building Services November 25, 2010 at 7:59 am #

    Hey how are you doing? I just wanted to stop by and say that it’s been a pleasure reading your blog. I have bookmarked your website so that I can come back & read more in the future as well. plz do keep up the quality writing

  12. CM November 25, 2010 at 12:43 pm #

    Aero Ninja,

    It was not because of the damage that Boeing vigorously pursued having the photos removed from the web. Frankly, (had Boeing let them remain on the web) the damage shown in the photos would have probably served to diminish the average person’s view of the severity of this event.

    If Boeing wanted the photos removed from the web solely because they put the company in a poor light or revealed a damaging design flaw, there is absolutely no way the Seattle Times or Ben Sandilands would have pulled them off the web. To do so would have probably been unethical for them as journalists. The photos were removed from the web because they contain detailed views of at least two specific design innovations developed with Boeing treasure. Boeing has every right, and even an obligation to their shareholders to do everything in their power to ensure this intellectual property is not handed to a competitor free of charge.

    Maybe this will help: If the New England Patriots took extreme, even legal action to ensure a lost copy of their playbook was not published on the web (and believe me, they would), I highly doubt you would suggest it was because the Patriots had something damaging to their public image in the playbook – You would logically know it was for competitive reasons. Given that Boeing has not worked in this way to have other leaked and damaging photos removed from the web (remember the barrel gap?), I would think you and others should be able to apply better critical thinking skills to this situation.

    As for Mr (Ms?) Phillips post, I take extreme exception to the suggestion Boeing would willfully fool themselves into believing a potentially dangerous problem is actually not a problem at all. To make such a suggestion when there’s absolutely no evidence this has ever happened, nor any evidence it is happening now is extraordinarily irresponsible. The only thing we know about Boeing policy on public information is they do not tell the public everything they know. I know this is irritating to us curious enthusiasts, but the last time I checked this was not illegal, unethical, dangerous, or in any way a reason to believe the worst of what history says is a world-class company.

    Cheers!

    CM

  13. Old guy November 25, 2010 at 7:00 pm #

    I hope that Boeing is thankful (apropos to the day) for CM and other apologists.

    I suppose that rolling out the ‘potemkin’ liner was quite right as far as CM was concerned. He, or she, ought to remember that, at the same time, Boeing, yes our great company, was touting that things were almost snapping together as planned, first flight was right around the corner, AND were just minimally hinting that there might be problems looming (all the while, they took a shell, essentially, and rolled it out to much fanfare).

    Now, if that were the only problem, CM would be right. But, we have several instances of the manager guys talking some imminent milestone attainment, with much confidence, just days (hours?) before there was a problem announced.

    Now, comments may go overboard sometimes. Yet, those guys who do the public dancing are not without their faults. Unfortunately, it is to those types that engineers, and others who know what is what, report.

    How did business descend to this idiocy? Plus, those dancers make oodles while doing things like jousting with the mere workers who, for the most part, are doing a better job than the dancers. Strange thing, indeed.

  14. interior designer toronto November 28, 2010 at 6:46 pm #

    Let’s do some cyber monday shopping tonite.