Testing Techniques
Usability Report Tips

by Chauncey Wilson
Usability Interface - August 1997

Have you ever wondered about reports of usability tests? How much time does it take to write one? What should you keep in mind when designing and writing the report? Here are some rules of thumb that I use:

1. Always provide your sponsors (clients) with a date for report availability and make sure that the report is ready. Make sure that you schedule time in for report writing (this seems obvious, but is not always done).

2. Target your report for a particular audience. Developers like as much detail as possible so a long report with the problem, rationale, and possible solution in a table format would be appropriate. If you are giving a 15-minute talk to the senior management group, then a short report (to accompany a 15-minute presentation) with an executive summary and brief details of key conclusions would be more appropriate than a long report.

3. For a detailed report, consider using one long table to list usability problems. If you use one long table, you can add/delete columns to it for sorting by different attributes. For example, if you have a column that numbers items from 1-N, a column that lists the object where the problem occurred (for example, the menu or a specific dialog box), and a column with a priority, you can sort by priority or object and use the column with numbers to get back to your original ordering. Using one long table, you can quickly copy and paste and sort and create custom version of the report. You could also use a code for which design principle was being violated and sort on principle.

4. A little tip that has worked from me when I have little time to analyze data is to get large Post-It notes and write one usability problem on each Post-It note as I am observing a participant or reviewing a product, then using affinity diagramming to quickly categorize the problems. This reduces the time it takes to organize data for a report (you don't have to excerpt things from your notebook or a logging tool. (Thanks to Mary Beth Butler of Lotus for this tip).

5. Providing the rationale or principle that underlies the problem is more convincing that a person's opinion. Whenever possible, I always try to provide a basis or rationale for a problem. Too often usability reports are just long lists of problems.

6. There is a bias toward reporting problems in usability reports. I think that it is useful to have a section in a report that describes what users like about a product. I've observed sessions where users like a number of things about a product, but the consultant's report focused only on the negative.

7. If there is a clear solution to a problem and you can come up with an alternative screen design, consider putting an illustration of your proposed solution in the document. Make sure that your audience is willing to accept some outside design recommendations though.

8. Ask your sponsors what they would like to see in a usability report. Outline the major sections of the report in your usability proposal. Ask if the team wants the raw data that supports the problems (make sure that you do not violate privacy when you do this).

9. Ask for feedback on your report. Usability specialists need feedback too.

10. Be quantitative whenever you can, even if it is to have a table showing the number of low, medium, and high usability bugs. If you did some think aloud studies or field interviews, you might note the number of people who have a particular problem (for example, of the 10 people interviewed, 9 experienced PROBLEM X) at least once during the testing or field interview).

 

Go to STC Society Web Site