Patterico's Pontifications

12/31/2007

NASA Releases NAOMS Airline Safety Study

Filed under: Air Security — DRJ @ 4:08 pm

[Guest post by DRJ]

NASA released today its controversial National Aviation Operational Monitoring Service (NAOMS) report, a study of commercial airline safety that was previously deemed too disturbing to release because it would upset travelers and hurt airline profits. For several reasons, the Houston Chronicle was not impressed:

“NASA begrudgingly released some results today from an $11.3 million federal air safety study it previously withheld from the public over concerns it would upset travelers and hurt airline profits. It published the findings in a format that made it cumbersome for any thorough analysis by outsiders. Released on New Year’s Eve, the unprecedented research conducted over nearly four years relates to safety problems identified by some 29,000 pilots interviewed by telephone.

Earlier characterizations from people who have seen the results said they would show that events like near collisions and runway interference occur far more frequently than previously recognized. Such information could not be gleaned from the 16,208 pages posted by NASA on its Web site, however, because of information that was edited out. The data was based on interviews with about 8,000 pilots per year from 2001 until the end of 2004.

The NASA Web site shows formatted, printed reports that the space agency scrubbed to ensure none of the pilots who were interviewed and promised anonymity could potentially be identified. The data was posted as NASA officials began a telephone news conference, allowing no time to look at the material and ask them questions about it.

NASA did not provide documentation on how to use its data, nor did it provide keys to unlock the cryptic codes used in the dataset.

NASA Administrator Michael Griffin told reporters the agency typically releases information in Adobe System’s portable document format, known as pdf, which presents the information on formatted, printed pages. But there are dozens of reports available from NASA’s Web site about other subjects in Microsoft’s Excel data format, which would permit researchers to conduct a meaningful analysis more easily.

Griffin said NASA wanted to ensure that no one modified the survey results and circulated false data as NASA’s research product. He said even inexpensive optical character recognition software could convert the formatted reports. Such software can risk introducing errors in the data as it performs these conversions. “We’ve gone the extra mile with this data, and well beyond our original intentions,” Griffin said.

He dismissed suggestions that NASA chose to release the data late on New Year’s Eve, when the public is distracted by holidays and news organizations are thinly staffed. “We didn’t deliberately choose to release on the slowest news day of the year,” Griffin said.

NASA drew harsh criticism from Congress and news organizations for keeping the information secret. Rejecting an Associated Press request under the Freedom of Information Act, NASA explained that it did not want to undermine public confidence in the airlines or hurt airline fortunes. Griffin later overruled his staff and promised Congress that he would release at least some data by the end of the year.

NASA’s survey, the National Aviation Operations Monitoring System, was launched to see if a massive pilot survey would help pinpoint problems and prevent accidents. Survey planners said it was unique because it was a random survey, with an 80 percent response rate, that did not rely on pilots to take the initiative to report problems but rather reached out and interviewed them.

Griffin said NASA never intended to analyze the data it collected, but rather they planned on passing on its methodology to the aviation community. He said he had only looked at a few results, but that, “It’s hard for me… to see any data here that the traveling public would care about or ought to care about.” That would be up to others who chose to analyze the data, he said.

Pilots were asked how many times they encountered safety incidents in flight and on the ground, such as near-collisions, equipment failure, runway interference, trouble communicating with the tower and unruly passengers.”

Based on this article, it’s hard not to share the Chronicle’s disdain for NASA’s attitude. On what planet is it logical to believe the traveling public would not be interested in “near-collisions, equipment failure, runway interference, trouble communicating with the tower and unruly passengers”?

It seems others are concerned, too:

“Griffin outraged some NASA employees by saying the project had been poorly managed and its methodology not properly vetted. Survey experts who worked on it, however, said they used state-of-the-art industry techniques and carefully validated the results.

NASA’s handling of the matter prompted a congressional investigation and separate investigations by its inspector general and by a union representing NASA workers.

Jon Krosnick, a Stanford University professor who helped design the project for NASA, said the release of information was inadequate. “The data they released are intentionally designed to prevent people from analyzing the rates properly and are designed to entrap analysts into computing rates that are much higher than the survey really shows,” he said Monday.”

Here is the NASA NAOMS website that contains links to the study the Chronicle described as too “cumbersome for any thorough analysis by outsiders.”

This sounds like a job for the internet!

— DRJ

4 Responses to “NASA Releases NAOMS Airline Safety Study”

  1. It will be very interesting to see if the relevant committees in Congress hold hearings on this report in any meaningful manner.
    The good thing is, that they only spent $11.3M of our money on this. I think that’s less than their coffee budget.

    Another Drew (8018ee)

  2. One of the features of this study that is not being told is that the act of turning in a record of the “incident” makes you immune from FAA enforcement. This means that everyone in aviation has every reason to “Report” all of these incidents. Airlines gave out these report cards to their pilots, and instructed them to fill these out whenever they had doubts about what happened. Now we are going to make regulations from this, absolutely Ridiculous.

    Helidoc (81e63a)

  3. Just read the results of the free response question that ends at the end of the Section C landing and approach questions for commercial pilots. At least 3/4 of the pilots experienced fast, hard landings or had difficulty getting in trim for landing or had to make changes to flight computers that reduced situational awareness. It seems clear that approach and landing, especially at busy, hub airports with parallel runways, are the chokepoint where the lack of experienced controllers and overworked pilots combine to reduce airline safety.

    Roger Friedman (7b90fd)

  4. This article and the study are worse gobbygook than the council meetings at Winnipeg Beach
    Its impossible to decipher – even if you legalease
    Seems it was meant to confuse
    My mortgage papers from my real estate lawyer are easier to understand than this
    Go jump in the lake

    Worse than real estate law (308b36)


Powered by WordPress.

Page loaded in: 0.3031 secs.