Field Trials: Trials and Tribulations of a Field Visit

Usability Interface
Field Trials: Trials and Tribulations of a Field Visit

by Robi Gunn
Technical Wrier, Applied Innovation, Inc.
Central Ohio Chapter, STc
Reprinted from Usability Interface, Vol 5, No. 2, October 1998

Both feet first

I was excited. After two years of documenting an application, I was finally flying out to meet some real live users and observe how they used our product, and if I was lucky, how they used our documentation.

I dutifully and eagerly prepared myself for the visit. I read books and STC articles on field visits and questionnaires. I was on a quest, and dangerously close to realizing a dream. At last, I would be able to define my audience, and gage the usability of the online help and hard copy manual. I would finally get the answers to my questions directly from a group of users.

My goal has always been to write meaningful documentation, that is, meaningful from the users’ perspective. At work I had asked questions and tried to determine who that elusive average user is, and what information the average user likely requires. I never got a sense that this had been previously defined, or that anyone could really give me that vital information. So I did what most of us do; I guessed. I tried to imagine what the average user was like, and then write for that person. Now that I was going to meet some users, they could confirm that my guesses had been correct, or provide me with the information I needed to re-focus the documentation to fit their needs. This was the first and most important piece of the puzzle for me. From there, it would be smooth sailing with just a few adjustments and fine tuning (I hoped). I couldn’t wait to hear what they thought of the documentation.

Ground Zero

"What do you mean you didn’t know that there was documentation?" I said it calmly and politely, but that little voice inside my head was shrieking. How in the world had these people used our product for two years and not been aware that there was documentation?

Besides providing hard copy manuals and online help, we had just recently placed all our documentation on a web site and created a CD. It was a shock to me that they could have gone two years without realizing that there was documentation. More mysterious yet, they had documentation at their fingertips, literally one keystroke away. I would have thought that at some point during the two years, someone would have randomly (accidentally?) pressed the F1 key. In two years, no one was ever curious enough to wonder what that help menu was, or what that help icon on the screen might do if you clicked on it?

It felt like a bomb had been dropped. I had flown out to meet the users, observe them using the product, and interview them about the documentation to establish a baseline. I was acutely aware that I was now standing at ground zero. I put away my carefully prepared folder of questions and research. It no longer had any meaning. I quickly came to the conclusion that I would have to go backwards before I could go forward.

Plan B

So instead, I began to demonstrate the online help. Prior to the visit, an agenda had been established that included meeting with various departments. During each meeting I gave a presentation of the basic features of the online help: Contents page, Index, Full Text Search feature, and context sensitivity. I indicated what procedures were documented and how they would typically (still just my best guess) use the online help to answer their questions. I got a great response. Most of the users seemed happy to find out that there was documentation, and the initial reaction was that it was "pretty good stuff". I also got the names and phone numbers of users that I could contact at a future date, and I promised to send copies of the manual and CD. I also gave them the URL for our web site, but found that there was some question if they had access to it because of security on their firewall. I thanked them for their time and interest, and headed back home.

Meanwhile, back at the ranch...

On the flight back, I mulled over how I would present this information to my boss. I concluded that it was a step in the right direction to find out the truth, regardless of how painful that truth turned out to be. I had established a baseline; it was just several hundred feet farther back than I had previously assumed. I had re-learned that valuable lesson: never assume anything. I also concluded that you have to start somewhere, and at least now we knew where that starting point was. A very important benefit from the trip: I had established contacts for the future. We could continue our mission to provide documentation that the users wanted.

Back at home, I sent copies of the manuals and CDs. I wrote thank you notes to our hosts. Then I began to follow up.
I verified how manuals are ordered and shipped. I confirmed that we had our processes in place, and that they were being followed. No problems there.

I collaborated with our Training Department. When the class for this product was taught, I made arrangements to provide pizza during a working lunch so that I had an opportunity to introduce and demonstrate the online help. Online help is now a regular part of the course. At the end of class for any product, all students now receive a copy of the documentation CD. In addition, the Publications Department is now registering documentation to get a handle on ‘who’ has ‘what’ documentation and we have started sending questionnaires.

I am planning to follow up with the contacts that I made now that they have had a chance to use the documentation. I will probably retool the questions that I had developed for the original visit into a questionnaire to send to them. A repeat trip to follow up and show our commitment to our users wouldn’t be a bad idea either. What is more flattering to a user, or demonstrates more interest, than to seek input from them? And with that input, you really can write meaningful documentation, which is all I ever wanted to do.

Well, it’s about six months since I made my field visit. I’m not where I want to be, and I’m not where I expected that I’d be, but I am hopeful about the future. I have an old plaque that says: ‘The longest journey starts with the first step.’ I have begun.

Request for feedback

Even though the online help has an F1 (context-sensitive) key, a help menu at the top of the screen, and an icon on the tool bar, users failed to notice that there was online help. Does anyone have suggestions on what I could have done to make them notice? You can email me at robileeg@aiinet.com.