We report our best practices for conducting moderated remote usability tests. Like a step-by-step tutorial, we want to enable you to manage your own remote usability test. We evaluated the tutorial internally and other team members could reproduce a remote usability test with almost no transfer costs.

We chose GoToMeeting for facilitating the test, because it provides a great video and audio quality and it gives you the necessary control to confidently manage an interview.

Prepare the remote usability test

Choose tools for conferencing, screen recording and video editing

We recommend GoToMeeting ($49/month) as conferencing tool, see our blog post Pick the right tools for remote usability tests. Because GoToMeeting does not record the webcam for privacy reasons, we used Camtasia (93,50 €) to record the webcam screen. We also used Camtasia for editing the videos and create highlight videos, but you can also choose other screen recording and video editing tools.

Write a task-based interview script

The script should be a line-up of scenarios and follow-up questions that make the participants comfortable and resample a stereotypical workflow. If you need help, feel free to contact us (nicole.charlier [at] akquinet.de). Nate Bolt shared an interview script here http://remoteresear.ch/samples/moderator-script/. In contrast to Nate’s script sample, I prefer to start a task with a scenario description that helps the participants to get into the situation. Besides, I leave a lot of whitespace between the questions to note answers and observations.

Schedule test sessions

There are several tools that will help you scheduling the test sessions (e.g., doodle, PowWow, iCal, Outlook, etc.). Keep in mind that you need some time before the session to get ready and after the session to save the data and get ready for the next session.

Send an instruction mail to participants

This mail is very important, then it builds the foundation for an successful remote usability test. Explain the procedure and clarify technical requirements (high speed internet, webcam, microphone, speaker/headphone, etc.). Do not forget to make your prototype remotely accessible in advance.

Meet and deliver with GoToMeeting + Camtasia

Edit meeting and recording preferences

1. Launch GoToMeeting and go to “Preferences – Meetings” (cmd + ,).
2. Disable “View the Attendee List”. (Hereby, we want to prevent that the participants feel uncomfortable by a list of observers.)

   

3. Go to “Preferences – Recordings”.
4. Define directory for recordings and set reminder to convert recordings.

   

Schedule meeting

1. Leave the preferences panel and click “Schedule Meeting”. ( GoToMeeting offers also a web interface for scheduling a meeting. This tutorial bases on the desktop app.)
2. Login with your GoToMeeting Account.
3. Choose an appropriate subject and appointment.
4. We have only positive experience with the built-in audio conferencing, but we choose “Provide both” for the case that it does not work properly.

   

Customize invitation mails

1. Add instructions to the default invitation mail for participants, e.g. “Since you will share your screen with us, we recommend you to close all application windows in advance.”.
2. Invite observers (e.g., team members, clients, UX researchers, etc.) and tell them that they mute themselves when they join the meeting.

Start meeting

1. Go to “My Meetings” and login with you name and email.
2. Check if the attendee list is disabled for attendees. Right click on an attendee in the attendee list and check if tick is gone in front of “Allow Attendee List”. (Unfortunately, I had sometimes the problem that disabling the list in the general settings was not applied.) 

   

3. Turn off “On-Hold Beeps” and “Entry/Exit Chimes”.

   

4. Mute all other participants. (New attendees are muted by default, so observers can not disrupt the interview.)

   

5. Check if “Mic & Speakers” is selected and your webcam shared if you want to. (There is also a window with your webcam video.)

   

6. Send Webcam Request to participant. (Participants has to confirm.)

   

7. Make participant presenter to request screen sharing.

   

Start recordings

1. Launch Camtasia and start “New Recording…” for participant’s webcam (shift + cmd + N).
2. Select “Custom Region”, microphone icon and speaker icon.

   

3. You may have to install the “System Audio Component” in “Settings – Recording”, before you can select the speaker icon.

   

4. Click the red button to select a recording region.
5. Draw the recording region around the webcam in the GoToMeeting Viewer.
6. Do not move the GoToMeeting window while recording and keep the webcam image unhidden.
7. Click “rec”.

   

8. Start recording in GoToMeeting.

   

Conduct the usability test

1. Start by explaining the purpose of the test and what is going to happen. – “We test software, not users.”
2. We usually send the link to the prototype within the GoToMeeting chat.
3. While the interview, you can note highlight times. This makes it easier to create a highlight video later.
4. Ellen Beldner (June, 2010) shared her learnings about how to manage an interview: Adventures in remote usability, Part 2: GoToMeeting

End meeting and stop recording

1. Camtasia – Stop Recording (alt + cmd + 2).
2. Recording is imported automatically to current Camtasia project.
3. End meeting by closing the GoToMeeting meeting window.
4. You will get reminded to “Convert Recording”.

   

5. Select last recording and click “Convert Recordings”.
6. Get an espresso or tea while converting.

   

7. Open recordings folder and drag & drop file to Camtasia.

Combine and edit recordings

1. Open Camtasia and Adjust Canvas (right click in canvas).

   

2. Arrange clips on canvas by drag & drop.

   

3. You can also change the size of the video elements.
4. “Separate Video and Audio” of screen recording (right click on GoToMeeting recording).

   

5. Delete audio track of screen recording. (The webcam recording by Camatasia contains all audio sources. The webcam recording is also synchronous with the participant’s voice. This synchrony has the highest priority, because the participant’s mouth is visible and it appears odd if the voice is out of phase with the speaking lips.)

   

6. Synchronize webcam and screen recording per drag & drop.
7. Since you started the webcam recording before the screen recording, you have to play around how much delay suits best. Listen to the audio and try to map it to the screen interaction.

   

8. Export project (cmd + E).
9. We recommend to choose the highest quality for the mp4-file.
10. Now, you can continue to extract highlight clips or deliver the whole recording.

We would like to hear your best practices for remote usability testing and whether this tutorial is helpful.

If you have any questions or remarks, drop us a line: Martin Grund (martin.grund [at] akquinet.de)