Doing voice research is very much different from usual forms of user research. And yes we’ve learned it the hard way. We’ve turned our experiences into some tips and tricks. Some of the them might seem obvious, but believe us, they’re not. Use them to your advantage.

Pretest

Take time to explain that the participant is going to test a VUI.
Sounds logical, right? Well, it isn’t. Would you ever explain to a participant what a desktop computer is before running a user test? Probably not. Not everybody knows what a Voice Assistant is. Let alone how it works and what you can do with it. Explain this briefly and have the participants ask a few simple questions to the Assistant to get comfortable.
Emphasize that it’s alright to make mistakes.
Generally, people find it exciting to test VUI’s. Put participants at ease by saying that you invited them because of their limited experience and because you want to test the product now that it’s still new and flawed.
Don’t ‘think-out-loud’!
We always ask participants to think out loud because we want to understand what goes on in their head. VUI’s are an exception to the rule, for the simple reason that Voice Assistant are always listening.
Uncover the participant’s experience, expectations and opinion on VUI’s before showing the test material.
For an important part, this will explain the way they react to the VUI. People with more ‘voice experience’ know better how to give voice commandos for instance. Besides, understanding the participant’s opinion on VUI’s before the test, allows you to spot changes of attitude afterwards.

Determine when you want to ask questions.
Asking questions while observing is not a great idea, because the VUI is listening in and might think it’s fun to react :) Besides that, questions disturb the participant’s flow. Try to save up your questions and ask them afterwards.
The setting of the test matters more than ever.
When testing a VUI, the setting is particularly important. Why? Because the relevance of VUI’s heavily depend on context.
We can’t see what goes on in the Voice Assistant’s ‘head’.
Meaning you have to pay extra attention to understand what’s happening exactly and why certain things go wrong. For instance: does the Voice Assistant not understand the commando or does it not hear the commando properly?

Do you also want to get tips and tricks on the post test and reporting of your voice research? Part two of this article is waiting for you.

If you are looking for a deep dive on how to design human centered voice solutions, you should take a look at Eventbrite to see when our next voice design workshop is.