Tracking Usability Problems for a Project

Usability Interface
Tracking Usability Problems for a Project

by Chauncey Wilson
Reprinted from Usability Interface, Vol 6, No. 2, July 1999

A reader asks, 'I have a question regarding the tracking of usability problems for a project. In some of the larger projects that I have worked on, we end up with a long list of fixes from multiple sources (heuristic evaluations, usability evaluations, user comments, etc). Many of the comments tend to get buried in a long list or pushed aside by high priority items. How are some of the ways this has been dealt with in the past?'

Chauncey Wilson responds:

1. All the issues should be put into whatever bug tracking system is used for all other software problems. This doesn't guarantee that things will be fixed but does ensure that they can be tracked from a single location.

2. Usability bugs are often rated as less severe than programming bugs and thus get a low priority in the development cycle. Usability bugs can be as severe as programming bugs (for example, a usability problem can cause permanent loss of data just as a crash can cause data loss). Usability bugs should have a severity scale similar to what is in use for programming bugs. Parallel severity scales help put usability bugs on an equal footing with programming bugs.

3. Heuristic inspections often reveal many platform or corporate standards violations. These violations are difficult and time-consuming to put into bug tracking systems and many, by themselves, would not cause major usability problems. The big issue is how many minor problems become a big problem? A technique used by some companies is to rate any violation of platform or corporate standards as a high priority even if it is a minor problem.

4. Layout and consistency issues can affect perceptions of usability and influence buying decisions. I like to work some "before and after" examples of GUI design into lunchtime seminars and corporate presentations. This is a good way to enlighten management about the impact of usability bugs.

5. At the end of usability tests or heuristic inspections take some time to set priorities and get agreement about what will be fixed. This is something that I routinely do in heuristic inspections. It would be quite useful if you can get developers to watch your usability tests or interviews and have them involved in setting priorities. There could be some political problems here since usability is defining work for development; therefore, some buy-in is needed from development managers.

6. An extremely powerful technique for setting priorities is an impact analysis where you create a chart of problems with a quantitative indication of their frequency and seriousness (how much time people wasted making errors versus total task time for example. You can then prioritize usability problems according to their actual impact on user performance. An impact analysis generally requires a detailed videotape analysis and is time-consuming, but it also one of the most convincing methods for prioritizing usability problems.

7. Give a seminar on usability bugs and discuss a proposed severity scale with development managers. Show how usability problems can be just as severe as programming problems. For example, accidental deletion of data because of a bad confirmation message can be catastrophic. A misspelling on a menu bar may seem trivial but it can bring ridicule to a product and make it seem shoddy.

8. Make sure that you become involved in sessions where managers and developers decide what bugs need to be fixed and lobby for equitable treatment of usability bugs.