UX-Men: How To Get Overseas Users To Test Your Design Remotely

Many people know that usability testing can help them refine their website/app design effectively. However, they have not incorporated usability testing into their software development lifecycle because of their users are overseas. Doing overseas usability testing projects can be quite costly. If your website/app targets overseas users, especially consumers, you should run remote unmoderated usability tests to evaluate your design.

• Remote — The test participants are in remote locations. For example, you are in Hong Kong and your participants are in the US. The remote unmoderated usability test providers (e.g., UserTesting.com, WhatUsersDo.com) record the test sessions. You watch the videos to do analysis.
• Unmoderated — The test sessions are conducted without any moderators. You submit the task instructions (e.g., “find the nearest MTR station”) to the test providers’ websites. The participants read the textual instructions and do the tasks. Without moderation means that you can’t probe the participants to find out why they did the tasks in the ways they did. Even if the participants think aloud, they may not tell you everything you want to find out. However, remote unmoderated usability test is definitely worthy to do. I would compare remote unmoderated usability tests to quick quizzes which are for quick evaluations, and face-to-face usability tests to thorough examinations.

Tips to run remote unmoderated usability tests

• Participants — You can use the user panels which many remote usability test providers offer, or find your own test participants.

  – If the test providers’s user panels have participants that can represent your target users, you should use the panels. When those participants do the tests, they think aloud. When you can’t talk to the participants, thinking aloud is very important in helping you deduce your product’s design issues. If you use your own participants, they are not expected to know how to think aloud.

  – Test with a small number of participants first to maximize your flexibility. When doing face-to-face usability test projects, we recruit enough participants in advance (5-7 participants per user profile typically). Many test providers let you run a test project using even 1 participant. In other words, you should begin a test project by using a small number of participants (e.g., 2-3 participants). Analyse the test videos. Determine if you need to adjust the tasks, before testing with more participants.

• Task design — You submit to the remote usability test providers the activities that you want the participants to try.

  – Handle task dependency. For example, After the participants “find out the capital of Uganda” (Task #1), and then you want them to “find out how much the ticket would cost to fly to the Uganda capital from where you are” (Task #2). If the participants fail to do Task #1, they can’t do Task #2. To solve the task dependency problem, you need to change Task #2 to “find out how much the ticket would cost to fly to Kampala from where you are.”

  – Conduct pilot tests. You won’t be able to explain the tasks to the participants remotely, so you need to make the tasks as clear as possible. Ask your friends, who are unfamiliar with your product, to do the tasks. Do your friends understand what the textual task instructions need them to do correctly? Can they finish all the tasks in 15 minutes (a remote test usually takes 15 minutes)?

• Post-test questions — The remote usability test providers usually let you ask the participants a few questions about they finish doing the tasks.

  – If the test providers do not let you ask the participants questions before they begin to do the tasks, use the post-test questions to find out more about the participants’ goals, behaviours, requirements, likes/dislikes and perceptions.

  – Ask why. For instance, “Please rate the usefulness of my app to you. 1 = Not useful at all … 5 = Very useful. Please explain the rating that you gave.” Different people interpret “usefulness” differently. You need to know in what ways your app is useful to the participants.

After testing, remember to share

Many people, who are not end-user-facing, have never seen how the real users use the products that they make. However, their work can impact their products’ user experience enormously. For instance, designing an error message seems very straight forward. However, if the users do not understand the error message, they may fail to check out the goods.

Sharing the remote test videos with your personnel can help remind them that the users are real people. Your personnel’s development or design decisions can delight or disappoint real people.

This article was written by Ada Yuen, Senior Usability Consultant and Co-Founder of AddiThink. 

Ada Yuen on Startbase.HK : Ada Yuen
AddiThink on Startbase.HK : AddiThink