by Alice Preston, Usability Engineer, Bellcore
Reprinted from Usability Interface, Vol 5, No. 3, January, 1999

Early in 1998, I performed an informal review on Remote Usability testing tools. This article documents what I found, some new information, and the impossibility of being too formal about this field.

Like PCs and the Web, to mention just two of the more notable examples, remote-testing tools seem to be on a merry-go-round of quick improvement. All of the literature that I have read tends to reserve the term "remote testing" for the type of testing that involves the tester/viewer "live" at one site and sees what's going on at a remote site.

Participants ask questions live (via a phone connection, audio via Internet, or typed chat session). There are some usability testing products (such as ErgoLight) that enable you to test remotely when you cannot make an online connection, but they are not covered in this survey. These products are classified as Remote Control, Support Desk/Customer Service, Telecommuting, System Administration, and Video Chat tools. Many of the products have a Recorder and/or Playback facility, which is probably a natural extension of remote viewing.

Depending on how carefully they're controlling the transfer of changes versus the transfer of entire screen, the playback can be helpful. In general, playback is of more use for determining what order the actions were done than in deducing anything about duration. Depending on the size/capability of the machines, the resolution and number of colors in the display, and the intricacy of the application being tested, both the remote testing tool and its recorder can affect the performance. If the application is one that users expect to have lightning-fast response time, introduction of remote testing software might adversely affect your results.

Several products offer a version of their viewer coded as a plug-in for a Web browser. Some new products support Windows CE, which operate on handheld devices.

However, there is an overriding problem for many businesses -- the Firewall! Of the products I tried, only OpSession had set up a reserved port for their transmissions (I'm told this is standard for the video conferencing tools). All others would require adherence to both the clients' and testing company's security conventions before a connection could be made.

Resources

Article on Remote-Control Software from September 1, 1998 issue of PC Magazine (http://www.zdnet.com/pcmag/features/remotecontrol/index.html) reviews the following products:

• Compaq Carbon Copy
• CoSession Remote
• LapLink for Windows
• NetOp for Windows
• pcAnywhere32
• RapidRemote
• ReachOut Enterprise
• Remotely Possible

Good overview of the subject, Jose Castillo and Rex Hartson's work, updated after UPA '97: http://hci.ise.vt.edu/~josec/remote_eval/index.html

Other approaches, such as ErgoLight, where you bundle the recorder with your software, user does testing, then sends log back to you for analysis see www.ergolight-sw.com

Comparison Matrix

Not all the products on this matrix were tried during the comparison, Some data is from marketing sources only.

Legend:

Package - Product ID / company name

Server platform - If a special Server version is required, the OS platform required

Viewer platform - What OS can run any version of the viewer (sometimes differently priced); some products run sessions with multiple viewers and no required server

Cost- Price as indicated on Web

Windows is a registered trademark of Microsoft Corporation.
UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company Ltd.

SIG Home | About the SIG | SIG Activities | Resources | Topics Newsletter |  Conference |  Bookshelf  | Toolkit  Comments or questions? Please send your email to <stcusability at sufficiently.com> © 1998-2004, Society for Technical Communication