About the Author
Bio.
|
STC UUX Community Newsletter |
 |
| Usability Interface |
The User Edit Method for Evaluating the Usability of Documentation
By Chauncey Wilson, The MathWorks, Inc.
A “user edit” (also known as a “usability edit) enables you to evaluate the usability of documentation (Schriver, 1991). Participants in a user edit study can either think aloud as they use the documentation to complete tasks or they can mark up the pages of the documentation to indicate where they had problems. The think-aloud protocols or marked-up pages are then reviewed for usability problems. The user edit report lists the problems and recommendations about how to improve the usability of the documentation.
Marshall Atlas (1981) was the first to describe the user edit as a type of verbal protocol method that could be used to understand the usability of documentation. His article in the IEEE Transactions on Professional Communication is considered a classic though few people actually seem to know about the method (it is seldom mentioned in HCI and UCD references). Atlas (1998) wrote a retrospective on his user edit method in the Journal of Computer Documentation where he discussed the question “Why hasn’t it [the 1981 user edit paper] had more impact on product documentation?” Other colleagues joined Atlas in the user edit retrospective and I would recommend that anyone interested in the history of documentation usability read the 1998 Journal of Computer Documentation in examining the user edit from the perspectives of five prominent documentation specialists (you can access it from the ACM Digital Library if either you or your organization has an account).
The user edit is perhaps the most powerful tool for finding usability problems in documentation. Soderston (1985) argues (and I agree with her) that this method finds significant problems in documentation, even after several technical peer reviews. She stated, “A user edit is a way of ensuring quality, not measuring it after the fact.”
The user edit method can be used to find problems like:
- Missing information
- Incorrect information (for example, the name of a sample file is wrong)
- Missing steps in a procedure
- Too many steps in a procedure
- Incorrectly ordered steps in a procedure
- Usefulness and usability of examples
- Ambiguous information
- Difficult terminology
- Usefulness of graphics, tables, charts, and other aids to text
- Key information that isn’t noticed
- A lack of examples or the presence of poor examples
There are two main approaches for conducting a user edit.
- The first approach has a participant think-aloud while using the product and the documentation.
- The second approach (the mark-up technique) involves having participants mark up printed documentation or paper copies of online documentation using a simple set of mark-up symbols or as they use the documentation individually.
Here are some basic instructions for the two approaches.
Think-aloud User Edit Procedures
A typical procedure for a think-aloud user edit study would be (Atlas, 1981; Soderston, 1985: Schriver, 1991):
- Choose a section of the documentation that you feel might have some usability problems or that involves new procedures. The user edit method is generally used with procedural documentation. The approach that I’ve used most often is to have the person read through each line of procedural instructions for a task. If information is needed during the task, it can be provided to the participant.
- Develop instructions for your participant. If you are doing a think-aloud user edit the instructions might be similar to the ones below:
”I would like to thank you for participating in a study about how we can improve our documentation for [insert product name]. Our goal is to have people use our documentation to work on some tasks and tell us how useful and usable it is. We are not evaluating your reading ability or skill. Rather we are looking at how the text can be improved for someone who will use it with this product. In this study we would like you to follow the procedures in the documentation and think-aloud as you perform a set of tasks.”
”We will ask you to read parts of the documentation line by line and perform the task as described in the documentation. [Author’s note, I have found it useful to ask the person to read the text line-by-line.] We want you to think aloud and tell us about anything in the documentation that is confusing, incomplete, ambiguous, missing, or just incorrect. You can tell us if the examples and graphics support the text. If you find things in the documentation that you feel is well done, please tell us about that as well.”
”Remember that as you work on the task, please say aloud anything that comes to mind while you are reading the documentation or doing the task itself. If you are quiet, I may ask you what you are thinking now. So please keep thinking aloud as your read and work on the task.”
Individual Mark-up Approach to a User Edit
In the late 1980s and early 1990s, our usability team at Digital Equipment Corporation used a variation of this method where participants would mark up documents using a coding scheme like the one in Figure 1.This coding scheme could be presented on paper or electronically so you could send users a list of tasks and pages in the documentation that you want them to read and ask them to mark-up the pages using codes similar to the ones in Figure 1.
Figure 1: Simple coding scheme for an individual user edit where a participant will be marking up documentation without a usability person necessarily present.
The codes in the mark-up technique must be simple and memorable since your participants might be alone when doing a user edit. It is good to provide contact information if participants have questions about the process.
Advantages of the User Edit
The user edit:
- Is most useful for evaluating the usability of procedural (as opposed to conceptual) information.
- Is useful for evaluating online and hardcopy documentation.
- Finds problems that even repeated peer reviews do not find.
- Is based on methods (thinking aloud and inspections) familiar to many usability practitioners.
- Provides input on text, graphics, and other visual components of documentation.
Disadvantages of the User Edit
The user edit:
- Requires participants to focus on the documentation while working with the product in a way that may not be completely realistic.
- Provides very detailed information, but the scope is narrow. You can not use this method to review an entire book; you need to select the critical section of documentation that is most important for product success.
- Generates a great deal of data that needs to be interpreted.
References
Atlas, M. (1981). The user edit: Making manuals easier to use. IEEE Transactions on Professional Communication, 24:1 (March): 28-29.
Atlas, M. (1998). The user edit revisited, or “if we’re so smart, why ain’t we rich?”. Journal of Computer Documentation. 22:3 (August). ACM Press: New York, NY. 21-24.
Schriver, K. A. (1991). Plain Language for Expert or lay audiences: Designing text using User Edit. (Technical Report Number 46) Pittsburgh, PA: Carnegie Mellon University, Communications Design Center.
Soderston, C. (1985). The user edit: A new level. Technical Communication, 1st Quarter, 16-18.
 |
|