Customer Satisfaction As Quality Metric
	Author's note: Thanks to Chuck Millar, Senior Editor II at Synopsys, for editing my article prior to submission. How can we as professional technical communicators best evaluate the quality of technical documentation? Some companies develop quality metrics by tracking support calls, others count spelling mistakes, and still others emphasize on-time delivery. Although all these methods are useful, for me the bottom line is customer satisfaction. Years ago when I worked for a voice-mail messaging company, I had a rude awaking. I hired a third-party vendor to query customers about the quality of the documentation produced by my department. The good news was that, according to the customers, our documentation was better than the documentation produced by our competition. The bad news was that the customers did not make much use of our documentation even though it was better. After studying these responses, we made quite a few changes in our approach, dropping some documents and enhancing others. I recall this experience whenever I hear discussions about quality and documentation. Because in today's economy I don't have the budget to hire a vendor to talk to customers, I recently decided to conduct an in-house survey to find out what our customers were thinking. First we set up a small committee to design and administer the survey. The committee came up with 22 questions regarding a customer's experience with our products, which products had been used most often, and the customer's preferred method of accessing our documentation. Although most of the questions were multiple-choice, some required comments. It was the comments that provided the most interesting information. The questions were simple and straightforward, such as “How can we improve the documentation to make it more useful to you?” After we posted the survey on our internal Web site in an area accessible only to customers, we sent an e-mail message to 500 randomly selected customers, encouraging them to take the survey. As an incentive, we announced that five $150 gift certificates would be awarded to randomly selected participants. In addition to making the regular survey available to customers, we also sent a modified documentation survey to the Synopsys application consultants (ACs), who rely heavily on our documentation in their interactions with customers in the field. This AC survey was modified to target documentation for specific products, thus providing results that would be especially useful to writers assigned to those products. Instead of learning about a generic category in need of improvement, some writers received specific comments about their own documents. After allowing a period of time for customers and ACs to participate in the surveys, the committee analyzed the results to identify a list of ways in which the documentation could be improved. Were there any surprises? Not really, at least not in a general sense. As anyone experienced in technical communication would expect, customers always want documentation that is technically accurate, clear, concise, and easy to access. But we did learn about some specific ways in which our documentation could be improved. Synopsys customers repeatedly asked for more and better code examples as their highest priority. Here are some of the requests for better examples:    Provide more realistic examples.    Add examples on how to use various features and commands.    Anticipate what might be unclear and provide examples for those concepts.    Provide more examples that show intricate problems that typically hold up engineers. Although some of my writers have engineering degrees, creating realistic examples is a difficult task that is best done by subject matter experts. Unfortunately, many of the subject matter experts do not get around to writing adequate code examples because doing so is not a high priority for them. A distinct benefit of this survey, then, is that it provides hard data that writers can use to convince their subject matter experts of the importance of examples to Synopsys customers. Further efforts to use customer feedback as a quality metric have been pursued by the committee on an ongoing basis. The committee plans to visit training classes to interview customers about documentation. The committee also plans to make site visits to customers to obtain their feedback about documentation. Partly as a result of the survey, some customers have contacted me independently about specific documentation problems they would like to see corrected. A few have actually agreed to review draft documentation. I routinely send thank-you gifts to customers who go to the trouble to provide this feedback. As you might expect, there still remains much to be done to improve the documentation. We are in a stronger position to achieve better quality, however, when we hear from our customers directly than when we assume we know what they want. Now that we know some of what our customers want, the committee will conduct the surveys annually to update the information. The committee will look for trends, develop solutions to problems, and provide writers with concrete information they can use to improve the quality of the documentation to meet the needs and wants of customers. Customer feedback thus provides an excellent metric for the quality of documentation. In addition, it tells us precisely where we need to improve.
	About the Author: Diane Davis, an STC senior member of the Silicon Valley Chapter and a member of the Bay Area Technical Publications Managers (BATPM), is Director of IC Implementation Technical Publications for the Corporate Applications and Marketing Group at Synopsys, an electronic design automation company, where she manages a staff of 30 writers, editors, production specialists, and managers. Diane holds a B.A. in English from Carnegie-Mellon University, an M.A. in Education from Glassboro State College, and a certificate in Managing Technical Publications from the University of Santa Cruz Extension. Back to top