The outage of a computer system used by airline pilots to file flight plans in the U.S will likely prompt a closer look at a $2.4 billion telecommunications system that has grappled with numerous problems in the past.
The U.S. Federal Aviation Administration (FAA) offered few details Thursday about the exact nature of the glitch, which caused major delays and flight cancellations in airports across the country. But in a statement, the agency blamed a "software configuration problem" within the FAA Telecommunications Infrastructure (FTI) in Salt Lake City.
That problem brought down a system used mainly for traffic flow and flight planning services for about four hours this morning. The flight management system -- it's called the National Airspace Data Interchange Network (NADIN) -- was affected because it relies on FTI services to operate, the FAA said. There was no indication that the disruption was the result of a cyberattack, the FAA said.
FAA experts were investigating the outage and meeting with Harris Corp., the company that manages FTI to "discuss system corrections to prevent similar outages," the agency said.
In an e-mailed statement, a Harris spokesman said the company is working to "evaluate the interruption" to prevent future outages. "FTI has proven to be one of the most reliable and secure communications networks operating within the civilian government. Safety and security is our highest priority," the company said.
A spokeswoman for the Professional Airways Systems Specialists (PASS) union, which represents more than 11,000 FAA employees, told Computerworld the problem arose when scheduled maintenance on FTI in Los Angeles corrupted a router in Salt Lake City. A back-up router that should have kicked-in when the primary router went down failed to do so, resulting in the widespread outage, she said.
The $2.4 billion FTI program was introduced by the FAA in 2002 to replace seven FAA-owned and leased telecommunications networks. It provides a range of voice, data and video communication services for operations and mission support at more than 4,000 FAA and Defense Department facilities, according to Harris. The FTI network provides switching and routing services, as well as centralized infrastructure security monitoring services for the FAA.
An audit of the program released by the FAA's Inspector General last September cited concerns over delays in the project's implementation and doubts about the promised cost-benefits the network was supposed to yield. The report also noted several technical problems that had caused unscheduled outages to air traffic control operations.
"In some cases, these outages have involved simultaneous loss of both primary and back-up FTI services, which not only disrupts air travel but also creates potential safety risks," the inspector general report warned, pointing to several incidents in recent years.
On Sept. 25, 2007 for instance, all FTI services were lost at the Memphis Air Route Traffic Control Control Center (ARTCC), disrupting air traffic control for several hours and causing 566 flight delays, the report said. The problem stemmed from a "catastrophic failure" of an optical network ring that was supposed to offer built-in fault tolerance. The FAA was vulnerable to the same issues at Atlanta and in Jacksonville.
Another incident occurred on Nov. 9, 2007, when all primary and alternate FTI services were lost at Jacksonville, resulting in 85 flight delays. "We also found that when FTI outages occur, the services are not always restored within contractual timeframes," the inspector general's report said. In some cases, where services are supposed to be restored within three hours, Harris took twice as long to fix the problem. "Several areas remain critical watch items for decision makers as FAA moves forward with FTI," the report said.
Others have been critical of the program as well. PASS, for instance, has in the past voiced concern over safety and efficiency issues related to FTI.
PASS spokeswoman Kori Blalock Keller said the FAA needs to hold Harris accountable for the problems. "If they are going to provide service, we need to make sure they are reliable and they
FTI combines all the FAA eggs into one basket. When a T3 fails, the sky falls. MCI and ATT had a great system/network. That is all gone.
Post new comment