Usability test report requirements

I'm in a discussion with some colleagues about the requirements of a good usability test report. I don't get too wrapped up in the details of reports, since I'm usually the only one who reads the reports. I'm more concerned about the content of the presentation and discussion. I often don't send the report out in advance, since I've sometimes had project managers take the report and decline the meeting. It depends on who I'm dealing with. The content of the presentations contain the basic findings of a study, but the conclusions are written specifically for the audience. I try to connect the work to their own concerns, which are different by area: development, marketing, operations, finance, and so on. Sometimes the most valuable part of the exercise is the discussion following the presentation of results. It tends to surface a lot of the assumptions, attitudes, and opinions of the people receiving the report, and gives me a chance to respond to a lot of concerns.

The funniest requirement for a usability report was given to me by a previous manager, thankfully long departed. I finished a test and report and told the manager that I was going to set up a meeting with the project team to deliver the results. She said the project team didn't have time and that I should just send the report to them and skip the meeting. This team wasn't familiar with usability testing or usability reports so I said, "What if the project team has questions?" The manager replied, "Just write the report with enough detail so they won't have questions."

Official Google Blog: Goodbye to an old friend: 1-800-GOOG-411

Google will shut down its speech reco application GOOG 411. I had written about this service before. I don't have a smartphone, and this was a handy service when you were away from your computer. Google rarely hesitates to pull the plug on a service that isn't meeting expectations, so I guess this is no exception. Thanks to Phillip Hunter for alerting me to this story.

Summary of HFES conference in SF

I went to the excellent Human Factors and Ergonomics Society conference in San Francisco last week. The keynote address was delivered by pilot Chesley “Sully” Sullenberger. Some of the talk was motivational, and some was specific to the human factors of the design of airline cockpit alerts, crew training, and performance under stress.

The emergency landing in the Hudson River in January 2009 was illustrated by a simulation showing a map of the area and an animated path of the aircraft from takeoff to landing. Overlaid on the screen was a clock, an altimeter, and some of the voice recordings between pilot and tower. Some interesting details that the speaker mentioned:

• The entire incident until landing was less than four minutes. He formulated his plan in less than a minute.
• There is an SOP for a situation in which both engines are lost, but it’s three printed pages long, obviously written for situations where the aircraft is at a high altitude. Some SOPs are printed, some are electronic, and pilots have to know where to look to get the right SOP.
• He had never specifically trained to ditch a commercial aircraft in water. The only training he received was a theoretical classroom discussion about it years earlier.
• He had never experienced a catastrophic equipment failure in 29 years of commercial flying.
• He feels that he was relying on his experience, including his fighter pilot experience 29 years earlier, to land the aircraft.
• He didn’t realize that he’d done everything correctly until the investigation into the crash concluded two months later.
• He was unable to deploy the flaps as he wanted to due to a software misfeature that was known only to a few software engineers. This caused the plane to come in faster and at a steeper angle than he wanted. This caused more damage to the underside of the aircraft than was necessary, and contributed to an injury.
• Airlines are reducing training to keep costs down. They train only to minimum FAA requirements and no more. He predicts that training shortfalls will become obvious only in the future and in exceptional circumstances. In other words, disasters will have to occur and data collected before changes will be made to training requirements.
• Commercial flight simulators still do not train pilots in the scenario he experienced in Jan. 2009.

The conference was heavy on NextGen research for airline industry and the health care industry. I chaired a session on "Management perspectives on building UX departments," which generated a lot of good discussion. I took a tour of the Autodesk office. I even ran into some people from the rail industry from Australia. And, of course, San Francisco is an excellent city to visit.