Results on a Study of Usability Testing

STC Usability SIG Newsletter: Usability Interface
Results on a Study of Usability Testing

by Elaine Ostrander
Reprinted from Usability Interface, Vol 7, No. 2, October 2000

A study I conducted as part of my graduate work at the University of Houston shows that technical communicators find many benefits in usability testing of documentation but cannot quantify them. The study’s purpose was to identify an exact return on investment figure that could be used to convince otherwise unwilling management to initiate usability studies. While the data failed to produce such a figure, impact analysis indicated that the return on investment is probably high.

The study focused only on informal, "over-the-shoulder" (rather than laboratory) usability testing of documentation, not of software or products. Participants reported the amount of time and hourly costs spent on usability testing. Participants also choose benefits of usability testing they had experienced from a list of possible benefits.

Participants reported costs with a high degree of confidence and produced a qualitative list of benefits, but could not quantify those benefits. However, impact analysis demonstrates that the quantitative benefits may be large, and shows considerable promise for a future study in which participants are trained to track and tabulate the benefits with greater accuracy. There is a strong indication that the return on investment for performing informal usability testing of documentation is significant.

Further study on this topic will need to include training for participants on how to track and tabulate the value added by usability testing of their documentation.

If you would like to participate in such a study, either as a participant or as a trainer, please contact me via email at eostrander@lgc.com or by mail: 18102 Bambridge, Houston, Texas 77090.

Note: A complete version of this study is also available.