Usability Interface
Remote Usability Testing Tools

by Alice Preston, Usability Engineer, Bellcore
Reprinted from Usability Interface, Vol 5, No. 3, January, 1999

Early in 1998, I performed an informal review on Remote Usability testing tools. This article documents what I found, some new information, and the impossibility of being too formal about this field.

Like PCs and the Web, to mention just two of the more notable examples, remote-testing tools seem to be on a merry-go-round of quick improvement. All of the literature that I have read tends to reserve the term "remote testing" for the type of testing that involves the tester/viewer "live" at one site and sees what's going on at a remote site.

Participants ask questions live (via a phone connection, audio via Internet, or typed chat session). There are some usability testing products (such as ErgoLight) that enable you to test remotely when you cannot make an online connection, but they are not covered in this survey. These products are classified as Remote Control, Support Desk/Customer Service, Telecommuting, System Administration, and Video Chat tools. Many of the products have a Recorder and/or Playback facility, which is probably a natural extension of remote viewing.

Depending on how carefully they're controlling the transfer of changes versus the transfer of entire screen, the playback can be helpful. In general, playback is of more use for determining what order the actions were done than in deducing anything about duration. Depending on the size/capability of the machines, the resolution and number of colors in the display, and the intricacy of the application being tested, both the remote testing tool and its recorder can affect the performance. If the application is one that users expect to have lightning-fast response time, introduction of remote testing software might adversely affect your results.

Several products offer a version of their viewer coded as a plug-in for a Web browser. Some new products support Windows CE, which operate on handheld devices.

However, there is an overriding problem for many businesses -- the Firewall! Of the products I tried, only OpSession had set up a reserved port for their transmissions (I'm told this is standard for the video conferencing tools). All others would require adherence to both the clients' and testing company's security conventions before a connection could be made.

Resources

Article on Remote-Control Software from September 1, 1998 issue of PC Magazine (http://www.zdnet.com/pcmag/features/remotecontrol/index.html) reviews the following products:

Good overview of the subject, Jose Castillo and Rex Hartson's work, updated after UPA '97: http://hci.ise.vt.edu/~josec/remote_eval/index.html

Other approaches, such as ErgoLight, where you bundle the recorder with your software, user does testing, then sends log back to you for analysis see www.ergolight-sw.com

Comparison Matrix

Not all the products on this matrix were tried during the comparison, Some data is from marketing sources only.

Package Server Platforms Viewer Platforms Cost Comments
CU-SeeMe / White Pine "MeetingPoint" conference server runs on Windows NT 4.0, Sun Solaris Win95, Windows NT, and Mac OS $69 to buy CU-SeeMe online and download Allows connection to NetMeeting, Intel ProShare.

LapLink / Traveling Software Win95/ Windows NT Win95, Win98, Windows NT 4.0, 3.1, and CE $149 suggested Contact:
www.travsoft.com/products/

Look@Me / Netopia (applet:) Win3.11, Win95 (plug-in:) Win95, Windows NT, Mac, UNIX? Free Timbuktu Applet & plug-in; can observe from NT but not to NT.

Available from: www.netopia.com/www/look/
ldownload.html

NetMeeting / Microsoft Win95, Windows NT Win95, Windows NT Free Available from: www.microsoft.com/netmeeting/
OpSession / NetManage

 

Win95, Windows NT Win95, Windows NT, UNIX $65 Good support; use 256 colors to improve performance, Has plug-in (free--can be freely distributed with software). Contact: www.opsession.com
pcAnywhere / Symantec

 

Win95, Windows NT Win95, Windows NT, 3.1, DOS, also for Handheld PCs, Java and ActiveX $169.95, pcAnywhere CE (handheld)

$79.95, pcAnywhere for Win 3.1,

DOS $179.95 [packaged prices, less for downloads]

Contact:
www.symantec.com/pcanywhere/
index/product.html

Remote Desklink / Traveling Software   Win95 $49.99 Connects to one computer from another; password protected,
Tango / Syracuse University

 

  Java applets (uses centralized servers) Free More of a set of tools for developers to implement with than a marketable product.

Available from:
http://trurl.npac.syr.edu/tango/

Timbuktu / Netopia Win3.x, Win95, Win98, Windows NT 3.51 & 4.0, and MacOS.

Also available in Enterprise version.

Win3.x, Win95, Win98, Windows NT 3.51 & 4.0, and MacOS.

Also available in Enterprise version.

$129.95 for a "twin pack" (Win95, Win98, WindowsNT).

$88.95 for single Macintosh.

Enterprise pricing not published on Web site,

Formerly known as "Farallon" systems.

Contact: www.netopia.com/software/tb2/

Legend:

Package - Product ID / company name

Server platform - If a special Server version is required, the OS platform required

Viewer platform - What OS can run any version of the viewer (sometimes differently priced); some products run sessions with multiple viewers and no required server

Cost- Price as indicated on Web

Windows is a registered trademark of Microsoft Corporation.
UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company Ltd.

 

Go to STC Society Web Site