When Testing for Ease of Use and Testing for Functionality Diverge

Usability Interface
When Testing for Ease of Use and Testing for Functionality Diverge

by Erica J. Seidel
Software Test Engineer, Sony Research Labs
Reprinted from Usability Interface, Vol 5, No. 2, October 1998

Introduction

This article was inspired by a question I posed to two electronic communities on the merits of combining the functions of usability and quality assurance (QA) into one group. The first community was a mailing list primarily serving usability and human factors professionals; the second community was the comp.software.testing news group. This question spawned a discussion of the similarities and differences between usability and quality assurance functions, working environments, requirements, and behavior.

Positive responses to the idea of combining usability with quality assurance:

Of the fourteen people who participated in the discussion, only five were positive about the viability of a combined usability/QA group. Those with a QA perspective tended to respond more favorably to the idea than those with a usability outlook. One of the justifications for a combined group was that "a bug is a bug whether it's in the code or the user interface design." Another respondent mentioned that a joint usability/quality assurance entity could be a "one stop product life-saving station," where "synergy between the two units can help promote each other." A third viewed usability testing as an integral phase of the overall QA cycle, because an unusable product will be perceived as a low-quality product.

Negative responses and considerations:

The other nine respondents were opposed to the idea of integrating usability testing with functionality testing. Our discussion raised the following considerations:

1. The functions of usability and quality assurance are fundamentally different. The usability community is more concerned with how people interact with technology, while quality assurance professionals are more concerned with whether that technology works under different parameters. Whereas usability professionals seek to create an overall positive user experience with a product, the quality assurance focus is on ensuring that a product is free of functional deficiencies before it exits the development arena.

2. Since both usability and quality assurance are frequently treated as after-thoughts, combining them would mean less of each is performed. Usability and quality assurance functions are alike in that ideally they are both involved from the beginning of the product development process. However, in practice, they are both often relegated to the final stages of development. This is because budgeting and scheduling priorities tend to favor the engineering phase, while testing needs are fulfilled only afterwards. So combining usability with defect discovery would mean that both functions would have to share already-scarce time and resources. Also, any usability testing or consulting at this stage would probably be too late to be very useful.

3. The specificity of a usability lab environment is different from that of a quality assurance lab space. The usability lab environment should approximate the environment in which the product will be used as closely as possible. So, if a remote control device for a television is being tested, the usability lab would ideally be converted into a living room with comfortable couches, where participants could relax in front of the television as they use the remote control. A quality assurance lab environment, on the other hand, requires a specificity that is less situational and more functional. Quality assurance professionals use their lab space to simulate different conditions under which the product will be used. In the case of the remote control, a quality assurance testing lab might have several different models of televisions with which to test the remote control device.

4. Usability testing involves external observation; quality assurance testing does not. The activity of a usability lab involves participants who test the product, moderators who administer the test, and observers. The observers can be internal employees such as developers and designers, or people who are external to the company. This latter case is especially common when the usability group is a consulting company hired to convince high-level management of a client company that usability is necessary for product success. But quality assurance activities generally occur in the absence of external observation. If you visit a software testing lab, you are likely to find only QA professionals, working in a subset of the development organization.

5. The typical developer's reaction to usability suggestions is more disgruntled than the response to quality assurance reports. Developers tend to resent usability suggestions, especially if they come late, since implementing them can require expensive re-engineering of the product. On the other hand, developers are much less likely to resent functionality defects that the QA team has discovered. This is because programmers are primarily concerned with getting things to work, as opposed to getting things to work enjoyably; if their product is found to be breakable, they are eager to fix it.

One way to get usability suggestions taken more seriously by developers is for QA people who are sympathetic to usability problems to 'disguise' usability bugs as functionality bugs. The strategy here is that a usability team can accomplish much more by speaking the developers' language to get their point across; and in some organizations, writing up usability problems as ‘bugs’ achieves that goal.

6. The skills and backgrounds of usability people and quality assurance professionals are different. Usability professionals must be flexible in working with people from different backgrounds. Part of their role is to craft compromises between individuals with competing interests, such as end users, developers, and managers. Their academic backgrounds are most commonly in psychology, cognitive science, and/or engineering. Quality assurance, however, demands people who are meticulous, detail-oriented and intent on making things break. Many companies seek out quality assurance engineers with technical degrees, such as computer science. It could be said that good usability people are more people-centered, and successful QA people are more technology-centered. One of the difficulties in creating a combined usability/quality assurance entity is that it is rare to find people with abilities in both usability testing and system testing who are also eager to merge these skills.

Special thanks to: Joel Baskin, Randolph Bias, George Casaday, Barry Dorgan, Dayan Golden, Pittas Marios, Shane Melton, L. Darrell Ray, Marlene Semple, and Vladimir Trushkin.