Demonstrate Documentation Value-Add with NASA's Automated Requirement Measurement (ARM) Tool
	Do your technical information products provide value to the organization? How do you know? How do you demonstrate a concept such as “value,” especially as it relates to technical information, and how do you apply meaningful quantitative and qualitative measures to such an apparent fleeting concept?
Table of Contents	The Relative Nature of Value NASA’s Automated Requirement Measurement (ARM) Tool: It’s Real and It’s Free The ARM Tool: Background How the ARM Tool Works Customizing the Quality Indicators List How Tech Pubs Teams Can Use the ARM Tool Summary Tool Download Instructions
The Relative Nature of Value	Most of the research into the topic of demonstrating value-add for technical publications involves “translating” some applied effect (use of templates, usability testing, use of Information Mapping® principles, and so on) into a dollar value, which is the language of functional organizations as they communicate about the “bottom line.” However, the final translation of such applied effects is subject to scrutiny and interpretation due to the relative value of that assigned dollar figure. This is especially true for most technical publications functions. When it comes to tools that help us demonstrate the value-add technical communications professionals can bring to documentation projects, let's face it: there isn't much out there. Wouldn't it be great if we had a tool that could detect automatically when a document conformed to specific requirements? Wouldn't it be great if that tool checked documents automatically for conformance to an organization's style and quality standards? And, what if that tool could output statistical reports that could be used to quantify value-add? Best of all, what if that tool were available today and at no cost? Back to top
NASA’s Automated Requirement Measurement (ARM) Tool: It’s Real and It’s Free	There is one tool that shows a lot of promise to do just those things, it's available for free, and it was created by the folks who first put Man on the moon. It's called the Automated Requirement Measurement (ARM) Tool and was developed by the Software Assurance Technology Center (SATC) of NASA’s Goodard Space Flight Center (GSFC). I have been following the development of this tool since 1998 when it was my job to write detailed software requirements specifications and proposals for complex e-commerce applications. Back to top
The ARM Tool: Background	The SATC developed the ARM tool (Figure 1) at the NASA Goddard Space Flight Center as an early life-cycle tool for assessing requirements specifications that are written in natural language (versus the more mathematical and difficult formal language). The ARM tool was designed originally for project managers to assess the quality of a requirements specification document. The tool does not evaluate the correctness of any specified requirements but does help with “writing the requirements right”, not “writing the right requirements.” Figure 1: The NASA ARM Tool NASA identified ten fundamental quality characteristics of software requirements specifications that are closely related to indicators of specification strengths and weaknesses, which are referred to as “quality indicators.” A quality requirements specification must be: Complete Consistent Accurate Modifiable Ranked Testable Traceable Unambiguous Verifiable Valid. Refer to the Automated Requirement Measurement (ARM) Tool Abstract for more information on these ten fundamental quality characteristics of software requirements specifications. The SATC compiled lists of quality indicators that were classified into nine separate categories (Imperatives, Continuances, Directives, Options, Weak Phrases, Size, Text Structure, Specification Depth, and Readability Statistics). The individual components in each category are words, phrases, and sentence structures that are related to quality attributes that should be found in requirements specifications. The ARM tool evaluates six of those nine Quality-Indicator categories (Text Structure and Specification Depth are calculated from some of the other six parameters; the Readability Statistics are those found with Microsoft Word). The nine categories fall into two classes: those related to individual specification statements, and those related to the total specification. The ARM tool searches documents for the presence of these default Quality Indicators in each of the nine categories. Instances are output into the report file. Back to top
How the ARM Tool Works	The ARM tool searches each line of a document (in ASCII/text format) for specific words and phrases your organization or tech pubs team has identified as document Quality Indicators (your Quality-Indicators list should consist of words and phrases to be avoided because you want to track non-conformance, not conformance). Using these indicators, the ARM tool creates an ASCII file that includes three reports. This file contains a Summary Report, a detailed Imperative Report, and a detailed Weak Phrase Report. The ARM Summary Report includes the total number of times each Quality Indicator occurs in the requirements document (Figure 2). As you scroll down through the report, you'll see the statistical results for the six Quality Indicator categories for the document in question. For example, Figure 2 shows that in the Imperatives category, the Quality Indicator “shall” was found once; the Quality Indicator “must” was found two times; and the Quality Indicator “will” was found five times. Figure 2: ARM Summary Reporting Indicating Instances of Imperatives Because you can customize the Quality Indicators in each category, you may find that rather than analyze the document for good quality indicators, you'll want to analyze the document for quality indicators that do not conform to your publishing standards so you can locate them quickly in the source file (Word or FrameMaker, for example) and correct them. The Imperative Report lists the location within the source file of each specification statement identified by the tool and a copy of the specification statement. The Weak Phrases Report lists the location and specific sentence or phrase that contains indicators that are considered to be phrases that weaken the structural, grammatical, or rhetorical integrity of the document (Figure 3). Figure 3: ARM Weak Phrases Report When you scroll down through the report, you'll find the exact location of the quality indicators the ARM tool checked for. For example, Figure 3 shows the precise location of three Quality Indicators labeled as “Detail Incomplete Report” (two TBDs, for “to be determined” and one TBS, for “to be scheduled”). Requirements specifications should not contain such incomplete information, so if you structure your Quality Indicators to search for violations of your standards, the report outputs those instances where the standards have been violated. Because the ARM tool allows for user customization of these Quality Indicators, you can use the tool to search through any document (not just requirements specifications) to determine how well it conforms to your organization’s quality policies and standards. You can make cosmetic changes to the post-processed ARM Summary Report, such as adding your own organization's heading and perhaps an explanation of the purpose of the report (see Figure 4 for an example), but you cannot customize the ARM tool to create custom output, with the exception of adding your own quality indicators. Figure 4: ARM Summary Report with Customized Heading Information Back to top
Customizing the Quality Indicators List	What you've seen so far is a document analysis using the SATC default Quality Indicators. What if you want to add some of your own or only use some of the SATC default Quality Indicators? That’s not a problem as the ARM allows for user customization. The ARM tool contains all the documentation you need to customize the Quality Indicators list. Back to top
How Tech Pubs Teams Can Use the ARM Tool	Tech pubs teams can use the ARM tool to serve as a diagnostics measurement of how well documentation (authored by writers, engineers, software developers) conforms to the defined quality and publishing standards in your organization’s style guide. The Summary Report statistics are compiled and can be reported as “percent compliant with (Your Company) publishing standards” and sent to authors, with cumulative statistical reporting being sent to engineering management. Reporting this information can help reinforce the need for content providers to enroll in FrameMaker training, for example, or to review the style guide. To provide a feel for value-add, subject each document file to the ARM tool “scrub” as it is received prior to any technical publications treatment. Once the final draft is completed, subject the document file to the ARM tool analysis again. Compare those statistics to the initial draft of the standalone document. The difference between the initial statistics and final statistics (along with other information) represents the value add your process brings to product development. As content providers apply the standards outlined in your organization’s style guide, the differences between the before-and-after statistical measures decreases, which often translates to less time required for editing that document. Back to top
Summary	The ARM tool provides technical publications organizations with a method for quantifying value add when used in conjunction with a well-thought-out quality program. How the ARM tool is used to quantify value add is limited only by the imagination of those who choose to use it. Besides serving as a diagnostics tool and a statistical analysis tool, it gives technical publications managers yet another reporting mechanism with which to speak the language of the bottom line. Back to top
Tool Download Instructions	To download the ARM tool: 1. Go to the Software Assurance Technology Center website at http://satc.gsfc.nasa.gov/tools/arm. 2. Scroll down to the bottom of the home page and read the download instructions to download the tool. To get information on how the ARM tool works and the default quality indicators the tool uses, download the paper entitled “Automated Requirement Measurement (ARM) Abstract” at http://satc.gsfc.nasa.gov/support/PNSQC_OCT96/pnq.html. For more information on the project, see “Project Support and Outreach” at http://satc.gsfc.nasa.gov/support/index.html. Back to top
	About the Author: Donn Le Vie, Jr. is technical publications project manager for Intel’s Personal Client Architecture Component Division in Austin, Texas. He also manages Intel’s Technical Communications Council, a company-wide organization of technical writers, managers, content providers, human factors professionals, and usability experts. Donn is a frequent presenter at regional and international STC conferences. He can be reached at donald.s.levie@intel.com. Back to top