Remote Usability Testing Tools
by Alice Preston, Usability Engineer, Bellcore
Reprinted from Usability Interface, Vol 5, No. 3, January, 1999
Like PCs and the Web, to mention just two of the more notable examples, remote-testing tools seem to be on a merry-go-round of quick improvement. All of the literature that I have read tends to reserve the term "remote testing" for the type of testing that involves the tester/viewer "live" at one site and sees what's going on at a remote site.
Participants ask questions live (via a phone connection, audio via Internet, or typed chat session). There are some usability testing products (such as ErgoLight) that enable you to test remotely when you cannot make an online connection, but they are not covered in this survey. These products are classified as Remote Control, Support Desk/Customer Service, Telecommuting, System Administration, and Video Chat tools. Many of the products have a Recorder and/or Playback facility, which is probably a natural extension of remote viewing.
Depending on how carefully they're controlling the transfer of changes versus the transfer of entire screen, the playback can be helpful. In general, playback is of more use for determining what order the actions were done than in deducing anything about duration. Depending on the size/capability of the machines, the resolution and number of colors in the display, and the intricacy of the application being tested, both the remote testing tool and its recorder can affect the performance. If the application is one that users expect to have lightning-fast response time, introduction of remote testing software might adversely affect your results.
Several products offer a version of their viewer coded as a plug-in for a Web browser. Some new products support Windows CE, which operate on handheld devices.
However, there is an overriding problem for many businesses -- the Firewall! Of the products I tried, only OpSession had set up a reserved port for their transmissions (I'm told this is standard for the video conferencing tools). All others would require adherence to both the clients' and testing company's security conventions before a connection could be made.
Article on Remote-Control Software from September 1, 1998 issue of PC Magazine (http://www.zdnet.com/pcmag/features/remotecontrol/index.html) reviews the following products:
Good overview of the subject, Jose Castillo and Rex Hartson's work, updated after UPA '97: http://hci.ise.vt.edu/~josec/remote_eval/index.html
Other approaches, such as ErgoLight, where you bundle the recorder with your software, user does testing, then sends log back to you for analysis see www.ergolight-sw.com
Not all the products on this matrix were tried during the comparison, Some data is from marketing sources only.
|Package||Server Platforms||Viewer Platforms||Cost||Comments|
|CU-SeeMe / White Pine||"MeetingPoint" conference server runs on Windows NT 4.0, Sun Solaris||Win95, Windows NT, and Mac OS||$69 to buy CU-SeeMe online and download||Allows connection to NetMeeting, Intel ProShare.|
|LapLink / Traveling Software||Win95/ Windows NT||Win95, Win98, Windows NT 4.0, 3.1, and CE||$149 suggested||Contact:
|Look@Me / Netopia||(applet:) Win3.11, Win95||(plug-in:) Win95, Windows NT, Mac, UNIX?||Free||Timbuktu Applet & plug-in; can observe from NT but not
Available from: www.netopia.com/www/look/
|NetMeeting / Microsoft||Win95, Windows NT||Win95, Windows NT||Free||Available from: www.microsoft.com/netmeeting/|
|OpSession / NetManage
|Win95, Windows NT||Win95, Windows NT, UNIX||$65||Good support; use 256 colors to improve performance, Has plug-in (free--can be freely distributed with software). Contact: www.opsession.com|
|pcAnywhere / Symantec
|Win95, Windows NT||Win95, Windows NT, 3.1, DOS, also for Handheld PCs, Java and ActiveX||$169.95, pcAnywhere CE (handheld)
$79.95, pcAnywhere for Win 3.1,
DOS $179.95 [packaged prices, less for downloads]
|Remote Desklink / Traveling Software||Win95||$49.99||Connects to one computer from another; password protected,|
|Tango / Syracuse University
|Java applets (uses centralized servers)||Free||More of a set of tools for developers to implement with
than a marketable product.
|Timbuktu / Netopia||Win3.x, Win95, Win98, Windows NT 3.51 & 4.0, and
Also available in Enterprise version.
|Win3.x, Win95, Win98, Windows NT 3.51 & 4.0, and
Also available in Enterprise version.
|$129.95 for a "twin pack" (Win95, Win98,
$88.95 for single Macintosh.
Enterprise pricing not published on Web site,
|Formerly known as "Farallon" systems.
Package - Product ID / company name
Server platform - If a special Server version is required, the OS platform required
Viewer platform - What OS can run any version of the viewer (sometimes differently priced); some products run sessions with multiple viewers and no required server
Cost- Price as indicated on Web
Windows is a registered trademark of Microsoft Corporation.
UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company Ltd.