Voice Search and Command Heuristics for Mobile Devices

Thomas Wendt UX Strategy "

The following document represents a summary of voice search and command case studies, academic research, and commentary from the design community. Heuris:cs are meant to provide guidance and high level support for design decisions.

 I. Transmodality over multimodality

Multimodal systems are those with multiple modes of data input. Google Search mobile app, for example, enables users to enter search terms via keyboard or voicetwo modes for the same data input. Transmodal systems are also multimodal, but they take steps to ensure all modes of input complement one another and work in tandem. Systems that incorporate both search and command need to account for how different interaction modes will work together."

"
Multimodal"

Text Nikki, What do you want for dinner?

Transmodal"

II. Use images and icons to clarify events

The movement from voice input to text output can be difcult. Short text output is tolerable, but long lists can quickly pass a threshold beyond which the effort to accomplish a task outweighs the potential result. Images and icons can simplify lists and reduce the cognitive load users experience when reading large amounts of text. Voice is an invisible interface; designers need to materialize certain aspects when necessary."

"

Text-based lists are cumbersome and difcult to navigate. Imagery can provide a quick means to scan information without getting caught up in details. Icons provide behavioral short cuts that prompt users to pay attention to the most important content." "

III. Design for narrative dialogue

Voice search queries tend to be more narrative than keyboard queries. Instead of uttering keywords (restaurant, Indian, New York City), voice search users speak directly to the application (Find Indian restaurants near me or What are some Indian restaurants near me?). Results for narrative searches should be returned as part of that dialogue. Even if the app does not speak back to the user (with voice), the ways that it presents text and/or images is part of the dialogue."

"

Find Indian restaurants near me.

X X X X

IV. Design for broad searches

Voice search tends to be more useful (and is therefore used more often) when the intent is to perform a broad search and receive multiple results. An example would be a user who searches, What action movies are playing tonight? The intent is to receive multiple results and choose the most appealing option. While single result searches do occur (Cosmopolis), users tend to simply type them."

"

X X X X
What ac=on movies are playing tonight?

X X X X

V. Design for use in private places

The vast majority of voice searches are performed either in the home or in a semi-private place (car, ofce, etc.). Voice control behavior is not yet common enough for most users to feel comfortable executing searches or commands in public spaces, especially if dealing with potentially sensitive or private information."

"
Primary" " Secondary" "

VI. Minimize interaction options

Voice interfaces coupled with visual interfaces tend to work best when the visual elements are hyperfocused on the task at hand. For example, input screens should be minimaljust enough visual elements to show how to initiate the microphone and that the system is listening. Command screens should focus on communicating how to use voice for next steps."

"

Clutter distracts users from completing desired tasks." "

Minimal design and direction helps users focus." "

VII. Provide guidance for voice prompts

Designers should aim to provide elegant guidance for users at various stages of the experience. Given that voice is a new interface for most users, they need to be taught how to best take advantage of the benets voice offers. This guidance helps set expectations and promote optimal behavior."

"

VIII. Account for the gaze

Given that the mobile device is the sole data input recipient, users will likely be visually focused on their phone when interacting with the voice system. Designers should use that attention to their advantage and utilize the screen as the main point of feedback."

"
Feedback" "

Primary" "

Further Reading

Google Search by Voice: A case study" http://goo.gl/dnfrp" " Comparison of Voice Search Applications in iOS" http://goo.gl/tjmtm" " Speech User Interface Guide" http://goo.gl/qIooS" " Voice as a User Interface" http://goo.gl/vVWWf" " Gaze and Speech in Attentive User Interfaces" http://goo.gl/5Mx8L" " Mainstreaming Speech-Enabled Web Applications" http://goo.gl/AV38I" " The Human Interface" http://goo.gl/b1SR0"