 |
Musing on Metrics column |
|
The mission of the first documentation group that hired me was to produce “timely and accurate” documentation. Twenty-five years later, I see no need to argue the point. Timeliness is the most important process metric; accuracy, the most important product metric. Accuracy is paramount: If a document is inaccurate, nothing else about it–fonts, formatting, presentation, or page design–matters. How do we measure accuracy? Mechanically, it's easy to do: take the most relevant item (procedural steps, declarative sentences), determine the number of factual errors, and calculate a percentage of correct statements. (Keep the raw data, of course, so you can roll it up and track changes over time.) You needn't count every fact and error in a document. The auto industry turns out millions of units every year while advertising high quality products, but it would be prohibitively expensive to inspect every car thoroughly. Instead, they pull a few cars off the assembly line at random and tear them apart, looking carefully for defects. The Deming methodology of statistical process control is to sample (or “spot check”) a statistically valid subset of the total output to estimate quality. Thus, checking the accuracy of randomly selected pages gives a valid picture of overall documentation accuracy. I realize statistical sampling is contrary to the “read every word” culture of technical writing and editing. Nevertheless, the larger the document (and the larger the writing group!), the more sense it makes. How do you determine if the information is accurate? There are three ways. First, technical review by conscientious subject-matter experts verifies accuracy and identifies errors. Working closely with subject-matter experts, during both writing and review, is the most important step you can take to improve the quality of your documentation. Technical review should be a universal requirement; I'm always surprised that it isn't. You may be concerned that the more reviewers you have, the more review comments you'll have to deal with. Perhaps. But it's better to flush out errors before they reach your customers. Does the technical quality of your document deteriorate with increased inspection? I don't think so. If anything, the more reviewers you have, the greater your confidence level in the result will be. Another approach is to have your document verified by your test group. Based on my experience, you can probably get them a draft for the start of the test cycle. They can test the draft as part of their overall test strategy. Short of that, look at the test plans; testers are likely verifying the product's characteristics themselves. See what they find out and tell your audience about it. You say your engineers don't have time to talk to you, and your company has no test group? Then do it yourself. You can verify everything you write against the product by examining and using the product yourself. I know that can be a challenge, particularly when products change right to the end of the development cycle and beyond. But it's worth your while, for your readers' sakes as well as your own. In fact, I think the working definition of a “technical” technical writer is someone who can verify what they write. Not sure what to do? Combine all three approaches! The journey is the reward: Even if you never collect data on accuracy, the attempt to verify accuracy and flush out errors will drive you to do the most important single thing you can do to improve the quality of your documentation. |
|
Next time |
Order from chaos–a parable |
|
About the Author: Steven Jong, a Senior Member of the STC Boston Chapter, is a technical writer and writing manager. |