STC Usability SIG Newsletter
Usability Interface

Testing Educational Software with Children

by Yves Rannou

In May 2001, I conducted a study of Knowledge Forum, an educational software package for young users. Part of the study was a series of usability tests to reveal user interface issues, assess their impact on the quality of users’ interaction, and identify what was causing user failures while interacting with some of the features.

Knowledge Forum is a multimedia fully component-based Internet-savvy knowledge-building environment. It builds on core technology and adds powerful and important graphical functionality such as customizable views, scaffolds, and build-ons. In Knowledge Forum, children find notes that represent student ideas and questions. Intellectual interactions occur as students build on to notes, reference others' work, make solicited contributions, rise-above previous notes to create new syntheses, or make collections of related notes. The Backpack is a place in Knowledge Forum where students can place notes and read them at a later time. It is similar to a Your Favorites feature.

One-on-one usability testing sessions took place at the school, in a room separate from the classroom. In the room were the researcher and the participant. Each individual session lasted approximately 1/2 hour. Each session began with the reviewer greeting the participant. Prior to the session and on a different day, the participants, with the help of the researcher, completed an initial questionnaire. The answers to this questionnaire helped determine the candidate's profile. The information was used to aid in the final assessment.

Testing included a number of questions and scenarios, derived from activities users are most likely to want to do on Knowledge Forum (i.e. the user's goals), which addressed the study's main questions. Thinking aloud was encouraged throughout the session to attempt to reveal the thinking process while the user is interacting with the application

What Children Liked and Disliked

Overall, the children liked Knowledge Forum. When asked, “What do you like most about Knowledge Forum?” they answered:

• Painting
• Responding to others’ messages
• Searching on messages
• Writing messages
• Reading messages
• Deleting messages
  

When asked, “What do you like least about Knowledge Forum?” they answered:

• The Keyword functionality
• Takes too long to write a full idea
• When things get erased
• The Backpack

Some of the things that the children reported disliking on the initial questionnaire also caused them problems during the one-on-one usability testing. For example, children used a single finger when typing, making it extremely slow to write a note. Most participants did not look at the screen while typing; therefore, they do not get immediate feedback as to whether or not they made spelling mistakes. A few participants closed the application by mistake instead of closing the message.

Most participants had difficulty to report the frequency and the duration for which they use the computer at home.

For some of the participants, the duration for which they use particular software was the most problematic measure to report on.

What I Learned

Based on my understanding of first grade students, educational software asks young children to perform activities that requires a level of abstract thinking that makes it impossible for them to accomplish. Activities include searching a database, creating search queries (children of that age group have limited knowledge of the search capability and how it works), manipulating how the data is presented, and so on. Hence the difficulty in using the Backpack where children can save messages to read later.

Based on the initial questionnaire, users of that particular age group have a particular interest for applications that convey a sense of game. What's intriguing is that Knowledge Forum has very few pictures or images that would convey a sense of game, yet children enjoyed using Knowledge Forum. I recently observed a class of students talking about Knowledge Forum in the classroom.

I learned that Knowledge Forum is more than just software; it is part of the classroom, part of the kids' language, as a way of thinking about knowledge building.

Issues of testing with children

Whenever I conduct a study with children, I always make sure that I understand the context of use, the environment in which the application is used, and who is interacting with the children. In the context of a classroom, it is important to consider the role of the teacher. Teachers should be involved in the usability testing (or at least, they should give their input to the project—a use of a triangulation).

Proper ethics is also a concern. Usability testing with children requires parental consent. When parents want to take part to the study; their level of involvement during the session (if any) must be
carefully managed so as not to influence the child's behavior or decisions.

Ages and Stages

A product is best suited if it meets its objective and only if its target audience(s) can use it (and want to use it i.e. see value in it) and interact with. For example:

• For children, ages 6 to 7, a good product should consider limited reading ability, limited writing ability, limited typing ability (children cannot type using more than one finger—therefore, they need an application that does not heavily rely on typing). Lots of images, sounds, and a chosen theme (e.g., space) throughout the application that will appeal to children are essential. Games, like products, appeal to most children; children also like to change some aspects of the application, such as background color and text size.
  
• For children ages 8 and above, some of these abilities do not represent as much of a problem as with younger children. The level of complexity can increase as they become more familiar with the product. With younger children, the number of features should be kept at a minimum to avoid a sense of overwhelming.
  

For additional information about learning styles of children see www.penpages.psu.edu/penpages_reference/28507/285071390.HTML