Human

Interface
Guidelines
v1.7
 ©2013 Microsoft Corporation. All rights reserved.

Acknowledgement and waiver

You acknowledge that this document is provided “as-is,” and that
you bear the risk of using it. (Information and views expressed in this
document, including URL and other Internet Web site references, may
change without notice.) This document does not provide you with
any legal rights to any intellectual property in any Microsoft product.
The information in this document is also not intended to constitute
legal or other professional advice or to be used as a substitute for
specific advice from a licensed professional. You should not act
(or refrain from acting) based on information in this document
without obtaining professional advice about your particular facts
and circumstances. You may copy and use this document for your
internal, reference purposes. In using the Kinect software and Kinect
sensors described in this document, you assume all risk that your use
of the Kinect sensors and/or the software causes any harm or loss,
including to the end users of your Kinect for Windows applications,
and you agree to waive all claims against Microsoft and its affiliates
related to such use (including but not limited to any claim that
a Kinect sensor or the Kinect software is defective) and to hold
Microsoft and its affiliates harmless from such claims.

Microsoft, Kinect, Windows, and Xbox are registered trademarks, and

Xbox360 is a trademark, of the Microsoft group of companies. All
other trademarks are property of their respective owners.

Kinect for Windows | Human Interface Guidelines v1.7 2

Contents 4 Introduction
Learn about the Kinect for Windows sensor and SDK, and how
placement and environment affect its use

14 Interaction Design Tenets for Kinect for Windows

Learn about our design principles and how to design the best
input for the task

21 Gesture
Learn interaction design considerations for various types of gesture

46 Voice
Learn interaction design considerations for voice, including language
and audio considerations

60 Feedback
Learn about interaction design considerations for gestural and audio
feedback, text prompts, and more

77 Basic Interactions
Learn about the areas of the Kinect screen and user space, plus
details on engaging, targeting, selecting, panning and scrolling,
zooming, and entering text

118 Additional Interactions

Learn about distance-dependent interactions, and designing
for multiple inputs and users

135 Conclusion

Kinect for Windows | Human Interface Guidelines v1.7 3

 Introduction

Introduction
Welcome to the world of Microsoft Kinect for
Windows–enabled applications. This document
is your roadmap to building exciting human-
computer interaction solutions you once
thought were impossible.

We want to help make your experience with

Microsoft Kinect for Windows, and your users’
experiences, the best. So, we’re going to set
you off on a path toward success by sharing
our most effective design tips, based on our
long-term design, development, and usability
work. You’ll be able to focus on all those unique
challenges you want to tackle.

Keep this guide at hand – because, as we

regularly update it to reflect both our ongoing
findings and the evolving capabilities of Kinect
for Windows, you’ll stay on the cutting edge.

Before we get into design tips, it will help

to start with a couple of basics: a quick
introduction to the Kinect for Windows sensor,
software development kit (SDK), and Developer
Toolkit, and some environment and sensor
setup considerations.

Kinect for Windows | Human Interface Guidelines v1.7 4

 Introduction > Meet the Kinect for Windows Sensor and SDK

Meet the Kinect

for Windows
Sensor and SDK
The Kinect for Windows sensor and SDK
provide the ears and eyes of your application.
You’ll want to keep their capabilities in mind as
you design your application.

Kinect for Windows | Human Interface Guidelines v1.7 5

 Introduction > Meet the Kinect for Windows Sensor and SDK

How Kinect for Kinect for Windows sensor

Windows sensor, Provides raw color image frames from the RGB camera, depth image frames from the depth

SDK, and Toolkit work camera, and audio data from the microphone array to the SDK.

together Kinect for Windows SDK

Processes the raw data from the sensor to provide you with information such as skeleton
The Kinect for Windows sensor, SDK, and
tracking for two people, player tracking for six people, and word recognition from audio data
Toolkit work as a team to provide new and
for a given language. You can download the SDK free from www.KinectforWindows.com.
exciting capabilities to your application.

Kinect for Windows Developer Toolkit

Using the Toolkit is optional, but you can find code samples in it that show how to use features
of the SDK, as well as components such as Kinect Button and Kinect Cursor, which help you build
interfaces faster. Using the components in the Toolkit lets you focus on your unique problem, and
makes the user experience of your application consistent with other Kinect for Windows–enabled
applications. Throughout this document, whenever we reference a sample or control, it is in the
Developer Toolkit, also downloadable at www.KinectforWindows.com.

The following sections cover what this team of products does to bring natural experiences to your
application. When we say “Kinect for Windows,” we mean the “sensor” and SDK working together,
unless otherwise specified.

Kinect for Windows | Human Interface Guidelines v1.7 6

 Introduction > Meet the Kinect for Windows Sensor and SDK

What Kinect for Near mode depth

ranges sweet spot

Windows sees • Physical limits: 0.4m to 3m

0.4m/1.3ft
0.8m/2.6ft
physical limits

Kinect for Windows is versatile, and can see • Sweet spot: 0.8m to 2.5m
people holistically, not just smaller hand
gestures. Six people can be tracked, including
two whole skeletons. The sensor has an RGB
2.5m/8.2ft
(red-green-blue) camera for color video, and
3m/9.8ft
an infrared emitter and camera that measure
depth. The measurements for depth are
returned in millimeters. Default mode depth
ranges
The Kinect for Windows sensor enables a wide
sweet spot
variety of interactions, but any sensor has • Physical limits: 0.8m to 4m 0.8m/2.6ft
physical limits
“sweet spots” and limitations. With this in mind, (default) 1.2m/4ft

we defined its focus and limits as follows: Extended depth (beyond

4m) can also be retrieved but
Physical limits – The actual capabilities of the
skeleton and player tracking
sensor and what it can see.
get noisier the further you
Sweet spots – Areas where people experience get away, and therefore may 3.5m/11.5ft

optimal interactions, given that they’ll often be unreliable. 4m/13.1ft

have a large range of movement and need to • Sweet spot: 1.2m to 3.5m
be tracked with their arms or legs extended.
Angle of vision (depth
and RGB)
27°
Note that Near mode is an actual • Horizontal: 57.5 degrees 43.5°
setting for Kinect for Windows, and is
different from the various ranges we • Vertical: 43.5 degrees, with
27°
detail in Interaction Ranges, later in -27 to +27 degree tilt
this document. range up and down 57.5°

Kinect for Windows | Human Interface Guidelines v1.7 7

 Introduction > Meet the Kinect for Windows Sensor and SDK

Skeleton tracking

Kinect for Windows can track

up to six people within its
view, including two whole
skeletons.

Full skeleton mode

Kinect for Windows can

track skeletons in default full
skeleton mode with 20 joints.

Seated mode

Kinect for Windows can also

track seated skeletons with
only the upper 10 joints.

Kinect for Windows | Human Interface Guidelines v1.7 8

 Introduction > Meet the Kinect for Windows Sensor and SDK

What Kinect for Audio input

Windows hears The Kinect for Windows

sensor detects audio input
Kinect for Windows is unique because its single from + and – 50 degrees in
front of the sensor.
sensor captures both voice and gesture, from
face tracking and small movements to whole-
body. The sensor has four microphones that
enable your application to respond to verbal
input, in addition to responding to movement.
Microphone array

The microphone array can be

pointed at 10-degree increments
within the 100-degree range.
For more information about audio and
This can be used to be
voice issues, see Voice, later in this
specific about the direction
document.
of important sounds, such as
a person speaking, but it will
not completely remove other
ambient noise.

Sound threshold

The microphone array can

cancel 20dB (decibels) of
ambient noise, which improves
audio fidelity. That’s about
the sound level of a whisper.
(Kinect for Windows supports 20 decibels
monophonic sound cancellation, cancelled

but not stereophonic.)

Sound coming from behind the

sensor gets an additional 6dB
suppression based on the design
of the microphone housing.

Kinect for Windows | Human Interface Guidelines v1.7 9

 Introduction > Meet the Kinect for Windows Sensor and SDK

Directional microphone

You can also programmatically

direct the microphone array
– for example, toward a
track me!
set location, or following a
skeleton as it’s tracked.

Loudest source targeting

By default, Kinect for

Windows tracks the loudest
audio input.

Kinect for Windows | Human Interface Guidelines v1.7 10

 Introduction > Consider Sensor Placement and Environment

Consider Sensor
Placement and
Environment
The situation in which your Kinect for
Windows–enabled application is used can
affect how users perceive its reliability and
usability. Don’t forget to test your application
early (and often) with the kind of environment
and setup you expect it to ultimately be used
in. Consider the following factors when you
design your application.

For more information, see Choose the

Right Environment for Voice, and Best
Setup for Controls and Interactions,
later in this document.

Kinect for Windows | Human Interface Guidelines v1.7 11

 Introduction > Consider Sensor Placement and Environment

Will many people be moving around the sensor?

You’ll want an optimal setup and space where no one comes between the engaged user and the
sensor. For example, you might want an interaction area defined by using a sticker on the floor
to indicate where the user should stand, or by roping off an area so that people walk around.

Will you rely on voice as an input method?

Voice input requires a quiet environment for reliable results. If you can’t control ambient noise level,
try designing for user interaction closer to the sensor. If a noisy environment is unavoidable,
voice might work better as augmenting other inputs (but not as the sole input method).

How far back are your users?

Consider what they can comfortably see. Design your user interface to account for the distance
at which people will interact, within the range of the sensor. Users who are older or visually
impaired might need graphics or fonts to be larger.

What will the lighting situation be?

Lighting affects image quality in different ways. So, if you envision a particular design, you’ll
want to specify that users check their lighting; and if you expect a certain kind of lighting, you’ll
need to design around those limitations. For example, ideally the light will come from behind
the sensor. The infrared depth camera works in all lighting situations (even darkness), but better
in moderate light than in direct sunlight or full-spectrum lighting. Dim lighting is fine for depth-
sensing applications (such as using avatars); with large amounts of natural light, skeleton tracking
is less reliable. On the other hand, color image quality does require good lighting (for example, if
you’ll use green-screening).

Kinect for Windows | Human Interface Guidelines v1.7 12

 Introduction > Consider Sensor Placement and Environment

How will your users dress?

Items that drastically change the shape of a person wearing or holding them might confuse
skeleton tracking. For example, black clothing, as well as reflective items, can interfere with the
infrared camera and make skeleton tracking less reliable.

Where will the sensor be placed?

Because your users will be interacting with the screen, place the sensor above or below the
screen, to directly face the subjects it’s expected to track. Avoid extreme tilt angles. If you’re
using the Kinect for Windows controls and interactions we provide, the distance between the
user and the sensor that will provide the most reliable behavior is between 1.5 and 2 meters.
If your scenario involves simple motion or blob detection and doesn’t rely on skeleton tracking,
you can mount the sensor to the ceiling or at other angles.

Kinect for Windows | Human Interface Guidelines v1.7 13

 Interaction Design Tenets for Kinect for Windows

Interaction Design
Tenets for
Kinect for Windows
Kinect for Windows opens up a new world of
gesture design and voice design.

Kinect for Windows | Human Interface Guidelines v1.7 14

 Interaction Design Tenets for Kinect for Windows > Overall Design Principles

Overall Design
Principles
Before we go into to our design guidelines,
we recommend you first read this brief
section for some very important interaction
tenets. They reflect the best ways we’ve
found to employ gesture and voice, and to
avoid some thorny issues.

Keep the following in mind as you design

your interactions.

| Human
Kinect for Windows v1.7 | Human
Interface
Interface
Guidelines
Guidelines
v1.7 15
 Interaction Design Tenets for Kinect for Windows > Overall Design Principles

The best user experiences are Each input method is best at

context-aware. something and worst at something.
• Your UI should adapt as the distance • Users choose the input that requires the
between the user and the sensor changes. least overall effort for a given scenario.
• Your UI should respond to the number and • People tend to stick to a single input unless
engagement of users. they have a reason to change.
• Place your controls based on expected • Input methods should be reliable,
user movements or actions. consistent, and convenient – otherwise
people will look for alternative options.
• Make sure your interactions are appropriate
for the environment in which your • Switching input methods should happen
application will be used. intuitively, or at natural transition points in
the scenario.
• The further the user, the wider the range
of movement.
• The closer the user, the more and finer the
content, tasks, and gestures.
• Environment affects user input.

Confident users are happy users. The strongest designs come after
• It’s important to keep interactions simple, user testing.
and easy to learn and master. • Kinect for Windows enables a lot of new
• Avoid misinterpreting user intent. interactions, but also brings new challenges.

• Give constant feedback so people • It’s especially hard to guess ahead of time
always know what’s happening and what will work and what won’t.
what to expect. • Sometimes minor adjustments can make a
huge difference.
• Conduct user tests often and early, and allow
time in your schedule for adjustments to
your design.

Kinect for Windows | Human Interface Guidelines v1.7 16

 Interaction Design Tenets for Kinect for Windows > Strong Inputs

Strong Inputs
In order to provide a good experience and
not frustrate users, a strong voice and gesture
interaction design should fulfill a number of
requirements.

To start with, it should be natural, with an

appropriate and smooth learning curve for
users. A slightly higher learning curve, with
richer functionality, may be appropriate for
expert users who will use the application
frequently (for example, in an office setting
for daily tasks).

For more information about designing

for different distance ranges, see
Distance-Dependent Interactions,
later in this document.

Kinect for Windows | Human Interface Guidelines v1.7 17

 Interaction Design Tenets for Kinect for Windows > Strong Inputs

A strong voice and gesture interaction design should be:

• Considerate of user expectations from their use of other common input
mechanisms (touch, keyboard, mouse).
• Ergonomically comfortable.
• Low in interactional cost for infrequent or large numbers of users
(for example, a kiosk in a public place).
• Integrated, easily understandable, user education for any new interaction.
• Precise, reliable, and fast.
• Considerate of sociological factors. People should feel comfortable
using the input in their environment.

Kinect for Windows | Human Interface Guidelines v1.7 18

 Interaction Design Tenets for Kinect for Windows > Strong Inputs

Intuitive, with easy “mental mapping.” Easy to back out of if mistakenly started,
rather than users having to complete the
action before undoing or canceling.

Efficient at a variety of distance ranges. Appropriate for the amount and type of
content displayed.

Kinect for Windows | Human Interface Guidelines v1.7 19

 Interaction Design Tenets for Kinect for Windows > Strong Inputs

The Right Input Natural actions

Mode for Your Task If one interaction is better at

a given task, consider using
As you design your interactions, keep in mind that one instead of forcing
all interactions into one
all the input methods available to you, and the
input mode.
pros and cons each have to offer.

For example, for entering text, let people use their physical keyboard
or a touch interface instead of gesturing.

Do Don’t

Enable users to accomplish tasks Force users to use inappropriate input

quickly and intuitively. methods for certain tasks just because
they’re good for other tasks in the scenario.

Use each input mode for what it’s Require an input method that feels
naturally best at. forced, unnatural, awkward, or tedious.

Take user orientation and location Switch input modes for the sake of
into account so that input modes are variety or effect.
switched only when it’s convenient and
benefits productivity.

Kinect for Windows | Human Interface Guidelines v1.7 20

 Gesture

Gesture
This section begins by providing some
important gesture definitions and goes on to
recommend a number of important design
considerations for gesture interaction.

Kinect for Windows | Human Interface Guidelines v1.7 21

 Gesture > Basics

Basics
In this document we use the term gesture
broadly to mean any form of movement that
can be used as an input or interaction to
control or influence an application. Gestures
can take many forms, from simply using your
hand to target something on the screen, to
specific, learned patterns of movement, to
long stretches of continuous movement using
the whole body.

Gesture is an exciting input method to

explore, but it also presents some intriguing
challenges. Following are a few examples of
commonly used gesture types.

Kinect for Windows | Human Interface Guidelines v1.7 22

 Gesture > Basics

Innate and learned Innate gestures

gestures Gestures that the user

intuitively knows or that
make sense, based on the
You can design for innate gestures that people
person’s understanding of
might be familiar with, as well as ones they’ll
the world, including any skills
need to learn and memorize. or training they might have.

Examples:

• Pointing to aim
• Grabbing to pick up
• Pushing to select

Learned gestures

Gestures you must teach the

Wave to engage
user before they can interact
with Kinect for Windows.

Examples:

• Waving to engage
• Making a specific pose to
cancel an action

Kinect for Windows | Human Interface Guidelines v1.7 23

 Gesture > Basics

Static, dynamic, and Static gesture

continuous gestures A pose or posture that the

user must match and that
the application recognizes as
Whether users know a given gesture by
meaningful.
heart or not, the gestures you design
for your Kinect for Windows application
Be wary of designing for symbolic static gestures such as “okay”
can range from a single pose to a more that might carry different meanings across cultures.
prolonged motion.

Dynamic gesture
Slide to confirm Confirmed!
A defined movement that
allows the user to directly
manipulate an object
or control and receive
continuous feedback.
Pressing to select and gripping to move are examples of
dynamic gestures.

Continuous gesture

Prolonged tracking of
movement where no specific
pose is recognized but the
movement is used to interact
with the application.

Examples include enabling a user to pick up a virtual box or perform

a whole-body movement.

Kinect for Windows | Human Interface Guidelines v1.7 24

 Gesture > Gesture Interaction Design

Gesture
Interaction
Design
With Kinect for Windows, you can explore
the new and innovative field of gesture
interaction design. Here are some of our
key findings and considerations in making
gesture designs feel “magical.”

Kinect for Windows | Human Interface Guidelines v1.7 25

 Gesture > Gesture Interaction Design

Accomplish gesture Users should agree with these statements as they use gesture
goals in your application:
• I quickly learned all the basic gestures.
The users’ goal is to accomplish their tasks
• Now that I learned a gesture, I can quickly and accurately perform it.
efficiently, easily, and naturally. Your goal is to
enable them to fulfill theirs. • When I gesture, I’m ergonomically comfortable.
• When I gesture, the application is responsive and provides both immediate
and ongoing feedback.

Kinect for Windows | Human Interface Guidelines v1.7 26

 Gesture > Gesture Interaction Design

Design for reliability • If the gesture is too circumscribed, unique, or complex, there will be fewer “false
positives,” but it might be hard to perform.
Reliability should be a top priority.
Without reliability, your application will • If the gesture is too unspecific or simple, it will be easier to perform, but might
feel unresponsive and difficult to use, and have lots of false positives and/or conflicts with other gestures.
frustrate your users. Try to strike a good
reliability balance.

Do Don’t
For more information about false
positives, see Engagement, later in this
Teach users how to effectively perform Drive users to other modes of input/
document.
a gesture. interaction because of poor reliability.

Instill confidence so users can show Require such rigid gestures that users
others how to perform a gesture. develop superstitious behaviors, like
making up incorrect justifications for
reactions they don’t understand.

Teach users how to perform a gesture Use different gestures for similar actions
early in the experience so they can unless there is too much overlap
use it in similar contexts. between the gestures and natural
variation in users’ movement.

Consider the frequency and cost of Design multiple gestures that are
false activations. too similar.

Kinect for Windows | Human Interface Guidelines v1.7 27

 Gesture > Gesture Interaction Design

Design for Game mindset

appropriate user Challenge is fun! If a user is

in game mindset and can’t

mindset perform a gesture, then it’s

a challenge to master it and
do better next time.
If you’re designing a non-gaming Kinect for
Windows–enabled application, or a specific UI
item such as a menu, keep in mind that game
mindset is NOT the same as UI mindset.

As you design, keep conscious of the purpose

of your application and the nature of your
users, and design accordingly.
In game mindset, a silly gesture can be fun or entertaining.

UI mindset

Challenge is frustrating. If
a user is in UI mindset and
can’t perform a gesture, he
or she will be frustrated and
have low tolerance for any
learning curve.

In UI mindset, a silly gesture is awkward or unprofessional.

Kinect for Windows | Human Interface Guidelines v1.7 28

 Gesture > Gesture Interaction Design

Design for natural Do Don’t

interactions Allow people to interact from a distance. Try to force-fit gesture or voice on
existing UI that was designed for a
Gesture might provide a cool new method of different input method.
interacting with your application, but keep in
mind that its use should be purposeful.

For example, don’t simply

map gestures to a touch
interface.

Allow gesture to enable an Require gestures for tasks that could be

interaction or expression that other done faster and more easily by using
input devices can’t. another input method.

Center part of the interaction on Require that gestures be used for

user tracking and poses – things that productivity, speed, and precision tasks.
Kinect for Windows does best.

Provide multiple ways to perform a task

if environment or context might change.

Kinect for Windows | Human Interface Guidelines v1.7 29

 Gesture > Gesture Interaction Design

Determine user intent

and engagement Hello!

Determining user intent is a key issue, and

hard to do right. Unlike other input devices,
which require explicit contact from a user, or
only track a tiny area of a person’s body, Kinect
for Windows sees a person holistically. Kinect
for Windows users are constantly moving and
interacting with the world, whether or not they Part of determining user intent is recognizing when people want to first engage with the
intend to be interacting with your application. application. It’s tricky because there’s no physical medium such as a mouse for detecting intent
Your challenge is to detect intended gestures with Kinect for Windows.
correctly and avoid detecting “false positive”
gestures. Keep in mind that users in a social If the user must actively trigger engagement, be sure you demonstrate it at some point early in
context may alternate between interacting with the experience, or make the trigger clearly visible when it’s needed. Avoid long or cumbersome
the application and with other people beside engagement experiences, because they will become frustrating for returning users or users who
them or in the vicinity. The application must have already observed someone using the application.
be aware and considerate of the social context
of the users. Do Don’t

Provide feedback to increase confidence. Forget to provide feedback when a

gesture fails or is canceled.

For more information, see the

Provide a clear and simple way for Require users to modify natural
Engagement and Feedback sections,
people to engage and disengage. behavior to avoid interaction.
later in this document.

Recognize unique and intentional Miss, ignore, or don’t recognize the

movement as gesture or interaction. user’s gesture.

Ignore natural body movements Misinterpret natural body movements

(scratching, sneezing, etc.). as the proposed gesture.

Kinect for Windows | Human Interface Guidelines v1.7 30

 Gesture > Gesture Interaction Design

Notes

Waving is one method for determining user intent. (For example, Xbox users must
wave back and forth at least three times, rotating at the elbow, for the movement to be
recognized.) Waving feels natural, yet it’s a unique movement not often misinterpreted.
It also carries meaning to people as a way to begin an interaction.

Other possible ways to recognize engagement involve how close the user is to the sensor,
or a particular pose, such as what direction he or she is facing.

You can also choose to recognize when someone disengages as a cue to explicitly turn
interactability off, rather than using a timeout or other passive method. For example, when
a user turns away from the screen, take that as a cue that they’re no longer engaged. (But
be careful not to disengage people accidentally in the middle of a task.)

Kinect for Windows | Human Interface Guidelines v1.7 31

 Gesture > Gesture Interaction Design

Design for variability Gesture interpretation

of input Simply “asking users to wave”

doesn’t guarantee the same
motion.
Users’ previous experience and expectations
affect how they interact with your application. They might wave:
Keep in mind that one person might not perform
• From their wrist
a gesture the same way as someone else.
• From their elbow
• With their whole arm
• With an open hand
moving from left to right
• By moving their fingers
up and down together

It’s hard to scan for a broad variety of interpretations; instead, it’s

best to give users clear tips about the exact gesture you require.

Kinect for Windows | Human Interface Guidelines v1.7 32

 Gesture > Gesture Interaction Design

Make the gesture fit Do Don’t

the user ’s task Make the action and feedback parallel –

for example, users swipe left to scroll left
Require users to move a hand up to
scroll content to the left.
Logical gestures have meaning and they relate or move content to the left.
to associated UI tasks or actions. The feedback
should relate to the user’s physical movement.

Use logical movements that are easy Require abstract body movements that
to learn and remember. have no relationship to the task and are
hard to learn and remember.

Make the size or scope of the motion Require a big movement for a small
match the significance of the feedback result, like a whole arm swipe to move
or action. one item in a list.

Use big, easily distinguishable Use big movements for actions that
movements for important and less must be repeated many times through
frequent actions. an experience.

Kinect for Windows | Human Interface Guidelines v1.7 33

 Gesture > Gesture Interaction Design

Design for complete Here are a few things to remember when defining a gesture set:
gesture sets • Make sure each gesture in an application’s gesture set feels related and cohesive.
• Keep your users’ cognitive load low; don’t overload them with gestures to remember.
The more gestures your application requires,
Research shows that people can remember a maximum of six gestures.
the harder it is to design a strong gesture
set. So, we recommend that you keep the • Take cues from existing gestures that have been established in other Kinect applications.
number of gestures small, both so that they’re • Test thoroughly for “false positive” triggers between gestures.
easy to learn and remember, and that they’re • Use obvious differentiators, such as direction and hand state, to make gestures
distinct enough from one another to avoid significantly different, reduce false positives, and avoid overlap.
gesture collisions. In addition, if you strive to
• Make sure similar tasks use the same gesture language, so that users can guess or
be consistent with other applications, people discover gestures through logical inference or pairing. For example, pair a swipe right
will feel at home with the gestures and you’ll (to move content right) to a swipe left (to move content left).
reduce the number of gestures they have to
learn. You’ll also reduce the training you have
to provide. IF: THEN:

Kinect for Windows | Human Interface Guidelines v1.7 34

 Gesture > Gesture Interaction Design

Do Don’t

Differentiate direction changes. Risk gesture overlap by making the

direction of two similar gestures
the same.

Differentiate progression or path. Have two gestures that follow the same
path, especially if they’re in the same
direction.

Have clear and different start and Have vague and similar start and end
end points. points that result in different actions.

Kinect for Windows | Human Interface Guidelines v1.7 35

 Gesture > Gesture Interaction Design

Do Don’t

Design logically paired gestures to Define the conceptual opposite action

be consistent and follow the same of a defined gesture as something other
gesture language. than a pair or mirror of that gesture.

Notes

Think about the whole scenario. What does the user do after completing a gesture?
Might that action look like the start of an unintended gesture? Will it put them in a
natural position to begin the next logical gesture for your common scenarios?

Kinect for Windows | Human Interface Guidelines v1.7 36

 Gesture > Gesture Interaction Design

Handle repeating Do Don’t

gestures gracefully Design repeating gestures to be fluid,

without jarring starts and stops.
Let feedback get in the way and hinder
or pace the user’s rhythm. For example,
If users will need to perform a gesture people shouldn’t feel like they must
repeatedly (for example, moving through wait for an animation to finish before
they can repeat the gesture.
pages of content), you might encounter a
few common challenges.
Enable users to quickly get into a Design a gesture such that repetition is
rhythm of movement. inefficient.

Consider the movement of the whole Design the opposite gesture to resemble
repetition. Ignore the “return” portion the “return” portion of the first gesture.
of a repeated gesture if it will disrupt
the ability of the user to repeat the
gesture smoothly.

Kinect for Windows | Human Interface Guidelines v1.7 37

 Gesture > Gesture Interaction Design

Avoid “handed” Do Don’t

gestures Allow users to switch hands to reduce

fatigue.
Require specifically handed gestures;
they’re not discoverable or accessible.
Handed gestures are ones that can be
done only with a specific hand. They
Accommodate both left and right- Design as if you, the developer or
do not provide users with a truly natural handed people. designer, are the only user.
experience, are undiscoverable, and should
be avoided.

Kinect for Windows | Human Interface Guidelines v1.7 38

 Gesture > Gesture Interaction Design

Vary one-handed and Do Don’t

two-handed gestures Use one-handed gestures for all

critical-path tasks. They’re efficient
Use two-handed gestures for critical,
frequent tasks.
For variety and to accommodate natural and accessible, and easier than two-
interaction, you can design for both single- handed gestures to discover, learn,
and remember.
handed and two-handed gestures.

Use two-handed gestures for noncritical Require the user to switch between
tasks (for example, zooming) or for one- and two-handed gestures
advanced users. Two-handed gestures indiscriminately.
should be symmetrical because they’re
then easier to perform and remember.

Kinect for Windows | Human Interface Guidelines v1.7 39

 Gesture > Gesture Interaction Design

Remember that
fatigue undermines
gesture
Your user shouldn’t get tired because of
gesturing. Fatigue increases messiness, which
leads to poor performance and frustration,
and ultimately a bad user experience.

good gestures in between

natural movements awkward movements
(unique and purposeful)
For more information, see Multimodal
Interactions, later in this document.

Do Don’t

Offer gesture alternatives: for example, Require excessive repetition of a single

in a list of items, use different gestures gesture, such as page swiping to get
to navigate in small or big steps, jump through a long list. (So, avoid long lists
within the list, zoom in or out, or filter or use a more appropriate modality, such
to shorten the list. as voice search.)

Require users to assume uncomfortable

positions, such as holding their hand
above their head for a long time.

Kinect for Windows | Human Interface Guidelines v1.7 40

 Gesture > Gesture Interaction Design

Consider user Do Don’t

posture and Design a single gesture that works well

across all viable postures.
Design alternate gestures for the same
command, such as different gestures
movement ranges for seated and full skeleton mode.

User posture might affect your gesture design,

so consider where and how your users will use Keep in mind the normal and Design one gesture that works well in
your application. For example, sitting limits a comfortable ranges of interaction for one posture but poorly in another.
person’s possible movements. the human body.

Be aware of the loss of reliability if Require users to stand at distances or

you view a user from the side. Joints orientations where skeleton tracking
that are not in full view are placed in will be less reliable. Examples include
predicted locations by the skeleton users oriented sideways to the sensor,
tracking and aren’t always in the blocked by objects (such as a table),
correct position. out of visible range of the sensor, etc.

Kinect for Windows | Human Interface Guidelines v1.7 41

 Gesture > Gesture Interaction Design

Teach gestures Teaching methods

How to Hello!

and enable • A quick tutorial for

new users
gesture

discoverability • An indication of gesture

state throughout the
Whenever a gesture is available to interact interaction
with your application, you should find a way
• A visual cue or hint when
to communicate it to your user. Here are a user first engages Wave to engage
some options.
• A static image
• An animation
• A message or notification

Kinect for Windows | Human Interface Guidelines v1.7 42

 Gesture > Gesture Interaction Design

Be aware of Tracking movement

technical barriers Keeping arms and hands to

the side of the body when
If you’re using skeleton data to define your performing gestures makes
them easier to track, whereas
gestures, you’ll have greater flexibility, but
hand movements in front of
some limitations as well. the body can be unreliable.

Tracking speed

For very fast gestures,

consider skeleton tracking
speed and frames-per-second
limitations. The fastest that
Kinect for Windows can track
is at 30 frames per second.

Field of view

Make sure the sensor tilt and

location, and your gesture
design, avoid situations where
the sensor can’t see parts of a
gesture, such as users extending
a hand above their head.

Tracking reliability

Skeleton tracking is most

stable when the user faces
the sensor.

Kinect for Windows | Human Interface Guidelines v1.7 43

 Gesture > Gesture Interaction Design

Remember your Physical differences

audience For example, you should

account for users of various
Regardless of how you define your gestures, heights and limb lengths.
Young people also make,
keep your target audience in mind so that
for example, very different
the gestures work for the height ranges movements than adults when
and physical and cognitive abilities of your performing the same action,
users. Think about the whole distance due to differences in their
range that your users can be in, angles that dexterity and control.
people might pose at, and various height
ranges that you want to support. Conduct
frequent usability tests and be sure to test
across the full range of intended user types.

Kinect for Windows | Human Interface Guidelines v1.7 44

 Gesture > Gesture Interaction Design

Iterate
Finally, getting a gesture to feel just right
might take many tweaks and iterations.
Create parameters for anything you can,
and (we can’t say it enough) conduct
frequent usability tests.

Do Don’t

Design a gesture that works reliably Design a gesture that works for you
for your whole range of users. but no one else.

Design a natural and discoverable Build your application to be inflexible,

gesture. so it is hard to make adjustments.

Kinect for Windows | Human Interface Guidelines v1.7 45

 Voice

Voice
Besides gesture, voice is another input
method that enables new and natural-feeling
experiences.

Kinect for Windows | Human Interface Guidelines v1.7 46

 Voice > Basics

Basics
Using voice in your Kinect for Windows–
enabled application allows you to choose
specific words or phrases to listen for and
use as triggers. Words or phrases spoken as
commands aren’t conversational and might not
seem like a natural way to interact, but when
voice input is designed and integrated well, it
can make experiences feel fast and increase
your confidence in the user’s intent.

Kinect for Windows | Human Interface Guidelines v1.7 47

 Voice > Basics

About confidence • Try to strike a balance between reducing false positive recognitions

levels and making it difficult for users to say the command clearly enough to
be recognized.
When you use Kinect for Windows voice- • Match the confidence level to the severity of the command. For
recognition APIs to listen for specific words, example, “Purchase now” should probably require higher confidence
confidence values are returned for each than “Previous” or “Next.”
word while your application is listening. You • It is really important to try this out in the environment where your
can tune the confidence level at which you application will be running, to make sure it works as expected.
will accept that the sound matches one of Seemingly small changes in ambient noise can make a big difference
in reliability.
your defined commands.

Kinect for Windows | Human Interface Guidelines v1.7 48

 Voice > Basics

Listening models Keyword/trigger

The sensor only listens for

There are two main listening models for
a single keyword. When it
using voice with Kinect for Windows: using hears that word, it listens Kinect!
a keyword or trigger, and “active listening.” for additional specified
words or phrases. This is
the best way to reduce false
activations. The keyword
you choose should be very
distinct so that it isn’t easily Home
misinterpreted. For example,
on Xbox360, “Xbox” is the
keyword. Not many words
sound like “Xbox,” so it’s a
well-chosen keyword.

Always on,
active listening

The sensor is always listening

for all of your defined words Home

or phrases. This works fine

if you have a very small
number of distinct words or
phrases – but the more you
have, the more likely it is that
you’ll have false activations.
This also depends on how
much you expect the user
to be speaking while the
application is running, which
will most likely depend on
the specific environment
and scenario.

Kinect for Windows | Human Interface Guidelines v1.7 49

 Voice > Basics

Choose words and Distinct sounds

Don’t

phrases carefully Avoid alliteration, words

that rhyme, common Syllables in Vowels in Same words in
syllable lengths, common Alliteration Rhymes Common Common different phrases
When choosing what words or phrases to
vowel sounds, and using
use, keep the following in mind. (If each the same words in different
CAT MAKE SOMETHING MAIN ONE MORE TIME
KIT TAKE SOMEONE TAKE SHOW ME MORE
screen has its own set of phrases, these phrases. KEY PAIR INCREASE DATE PLAY MORE LIKE THIS
guidelines apply within a set; if the whole PEAR DECREASE FACE MORE RED

application uses one set of phrases, the

guidelines apply to the whole application.)
Brevity
Do Don’t
Keep phrases short (1-5
words). SHOW THE LEFT ONE SHOW ME THE ONE
ON THE LEFT

PUT IN CART PUT THIS ONE IN THE

SHOPPING CART

Word length
Do Don’t
Be wary of one-syllable
keywords, because they’re PLAY THIS ONE PLAY
STOP VIDEO STOP
more likely to overlap with
SHOW MORE SONGS MORE
others.
SCROLL RIGHT SCROLL
GO BACK BACK

Simple vocabulary
Do Don’t
Use common words where
possible for a more natural TURN UP MAX OUT
RED CRIMSON
feeling experience and for
FIRST INITIAL
easier memorization.

Kinect for Windows | Human Interface Guidelines v1.7 50

 Voice > Basics

Minimal voice prompts

Keep the number of phrases

or words per screen small
(3-6).

Go home, Go back,
Next page, Previous page

Word alternatives

If you have even more items

that need to be voice-
accessible, or for non-text 1 2 3
camera 2
based content, consider
using numbers to map to
choices on a screen, as in
this example.

User prompts

For commands recognized

with low confidence, help
course correct by providing
prompts – for example, “Did Did you mean “camera”?
you mean ‘camera’?” camrwoj

Kinect for Windows | Human Interface Guidelines v1.7 51

 Voice > Basics

Reduced false activation

Test and be flexible. If a Confidence level threshold

specific word always fails
or is falsely recognized, try
to think of a new way to
describe it.

You can require higher confidence levels (80% to 95%) – that is, having
Kinect for Windows respond only when it’s certain that the user has given
the correct trigger. This might make it harder to users to interact, but
reduce unintentional actions.

Acoustics
Test Test
Test your words and
phrases in an acoustic
environment similar to
VS
where you intend your
application to be used.

Triggering audio

Try using a trigger or event

to make Kinect for Windows
start listening. For example,
only listen when a skeleton is
detected in a certain area.

Kinect for Windows | Human Interface Guidelines v1.7 52

 Voice > Voice Interaction Design

Voice Interaction
Design
Generally, it’s best to avoid forcing people to
memorize phrases or discover them on their
own. Here are a few best practices for helping
users understand that they can use voice, and
learn what words or phrases are available.

Kinect for Windows | Human Interface Guidelines v1.7 53

 Voice > Voice Interaction Design

Visual voice prompts

Display exactly how users

must say voice commands.

“Show me the next one”

“Show me the next 20”

Listening mode

Indicate visually that the

application is “listening”
for commands. For
example, Xbox360 uses a
microphone icon.

User assistance

Display keywords onscreen,

or take users through a
beginning tutorial.

First say “Kinect”,

then say “Home” or “Next”

Kinect for Windows | Human Interface Guidelines v1.7 54

 Voice > Voice Interaction Design

Visual notifications

If there is an issue with the

microphone connection,
display an error icon and/
or message so the user can
address the problem.
! Microphone is not detecting
audio, please check connection

Audio prompting

If for some reason the user

might not be facing the The quick brown
screen, have an option for fox jumped over
the lazy dog
Kinect for Windows to read
the available phrases out loud.

Alternative input IF: THEN:

Voice shouldn’t be the only

method by which a user can
interact with the application.
Build in allowances for
the person to use another
input method in case voice
isn’t working or becomes
unreliable.

Kinect for Windows | Human Interface Guidelines v1.7 55

 Voice > Voice Interaction Design

VUI (voice user interface) bars and labels

As seen in Xbox360 interfaces, VUI bars and labels are a great way to indicate what commands are
available, especially if you’re using a keyword. In Xbox360, the user says the keyword (“Xbox”), and
then the VUI bar appears, containing phrases that he or she can choose from. This is a good way of
clearly indicating what phrases are available, without having them always present on the screen.

Kinect You can say:

“Home”, “Next”, “Back”

See it, say it model

The “see it, say it” model is one where the available phrases are defined by the text on the screen.
This means that a user could potentially read any UI text and have it trigger a reaction. A variation
of this is to have a specified text differentiator, such as size, underline, or a symbol, that indicates
that the word can be used as a spoken command. If you do that, you should use iconography
or a tutorial in the beginning of the experience to inform the user that the option is available,
and teach them what it means. Either way, there should be a clear, visual separation between
actionable text on a screen and static text.

Lorem ipsum dolor sit amet,

consectetur adipisiciing elit,
When mic icon is blue, sed do eiusmod tempor
you can say words in blue. incididunt ut labore et dolore
magna al qua.

Kinect for Windows | Human Interface Guidelines v1.7 56

 Voice > Voice Interaction Design

Choose the right Ambient noise

environment The sensor focuses on the loudest sound source and attempts to cancel out other ambient

for voice noise (up to around 20dB). This means that if there’s other conversation in the room (usually
around 60-65dB), the accuracy of your speech recognition is reduced.
There are a few environmental
considerations that will have a significant
effect on whether or not you can
successfully use voice in your application.

60-65dB =

Amplify that to the sound level of a mall or cafeteria and you can imagine how much harder it
is to recognize even simple commands in such an environment. At some level, ambient noise is
unavoidable, but if your application will run in a loud environment, voice might not be the best
interaction choice. Ideally, you should only use voice if:

• The environment is quiet and relatively • There won’t be multiple people speaking
closed off. at once.

Kinect for Windows | Human Interface Guidelines v1.7 57

 Voice > Voice Interaction Design

System noises and cancellation

Although the sensor is capable of more complex noise cancellation if you want to build that
support, the built-in functionality only cancels out monophonic sounds, such as a system beep, but
not stereophonic. This means that even if you know that your application will be playing a specific
song, or that the song will be playing in the room, Kinect for Windows cannot cancel it out, but if
you’re using monophonic beeps to communicate something to your user, those can be cancelled.

Distance of users to the sensor

When users are extremely close to the sensor, the sound level of their voice is high. However,
as they move away, the level quickly drops off and becomes hard for the sensor to hear, which
could result in unreliable recognition or require users to speak significantly louder.

Ambient noise also plays a role in making it harder for the sensor to hear someone as they
get farther away. You might have to make adjustments to find a “sweet spot” for your given
environment and setup, where a voice of normal volume can be picked up reliably.

In an environment with low ambient noise and soft PC sounds, a user should be able to
comfortably speak at normal to low voice levels (49-55dB) at both near and far distances.

Audio level detected

Distance of user to Kinect

Kinect for Windows | Human Interface Guidelines v1.7 58

 Voice > Voice Interaction Design

Social considerations
Keep in mind the social implications of your users needing to say commands loudly while using
your application. Make sure that the commands are appropriate for the environment, so you
don’t force your users to say things that will make them uncomfortable. Also make sure that
the volume at which they have to speak is appropriate for the environment. For example,
speaking loud commands in a cubicle-based office setup might be distracting and inappropriate.

Kinect for Windows | Human Interface Guidelines v1.7 59

 Feedback

Feedback
Whether you employ gesture, voice, or
both, providing good feedback is critical
to making users feel in control and helping
them understand what’s happening in the
application. This section covers some ways you
can make feedback as strong as possible.

Kinect for Windows | Human Interface Guidelines v1.7 60

 Feedback > Basics

Basics
It’s important, especially if your users are
standing at a distance and have little direct
contact with the interface, to take extra care
in showing them how their actions map to
your application. Also, because a gesture is
only effective and reliable when users are
in the correct visibility range, it’s important
to give feedback to them in case they don’t
know when they’re out of range.

Kinect for Windows | Human Interface Guidelines v1.7 61

 Feedback > Basics

Consider what users

want to know
As you design, imagine you’re a first-time user,
and ask yourself these questions.

For more information, see Engagement,

later in this document. • Is the sensor on and ready? • Am I in control?
• Can I engage now?

• What does the sensor see? • How much of me can the sensor see?
• Where’s the field of view? • Is my head in view?

Kinect for Windows | Human Interface Guidelines v1.7 62

 Feedback > Basics

• How many people can the sensor see? • When and where can I gesture?
• How do I know it’s seeing me and not
someone else?

Notes

Many of these questions can be answered by displaying a small User Viewer (visualizing
depth) on the screen to show what Kinect for Windows sees at any given time.

Highlighting players, hands, or other joints in the depth viewer might also be helpful.

You can also prompt people to move into the appropriate field of view whenever they’re
cropped, too close, or too far back.

Kinect for Windows | Human Interface Guidelines v1.7 63

 Feedback > Feedback Interaction Design

Feedback
Interaction
Design
This section gives tips for designing various
kinds of feedback, including selection states,
progress indicators, and other visuals, as well
as audio feedback.

Kinect for Windows | Human Interface Guidelines v1.7 64

 Feedback > Feedback Interaction Design

Best practices Make it clear what content the user can take action on, and how
There are several best practices that apply
whether you’re designing for gesture or voice. Differentiated controls
BUTTON LINK NORMAL TEXT
Use iconography, colors, or
tutorials to show users how Lorem
to differentiate between ipsum dolor
For more information, see Targeting controls they can activate, Lorem Lorem ipsum dolor sit amet,
and Selecting, later in this document. consectetur
text prompts for voice input, adipiscing
and other text and content.

Input suggestions

Use iconography or
tutorials to show users
what input methods are
available to them.

Gesture suggestions

Show what gestures are Press to select

available.

Visual feedback

Show cursor visuals if you’re

tracking a user’s hand.

Kinect for Windows | Human Interface Guidelines v1.7 65

 Feedback > Feedback Interaction Design

Command suggestions

Show what commands users

can speak.

You can say:

“Home”, “Next”, “Back”

Audio notifications

If there’s an error with a

Kinect for Windows state (for
example, a hardware or a
connection issue), show the
error message or state and a ! Microphone is not detecting
audio, please check connection

suggestion for how to fix it.

Display some change in the
visuals. Consider whether
you should recommend that
the user switch to alternative
input methods. For an example of how to do this, see the Sensor Chooser component
in the Kinect for Windows Developer Toolkit.

Kinect for Windows | Human Interface Guidelines v1.7 66

 Feedback > Feedback Interaction Design

Map feedback to gesture

Progress feedback

When a user controls

something by direct
manipulation, show the
progress in a way that
translates to the person’s
motion.
For example, if the user turns a page by a gesture, show the page pulling
up and turning over as his or her hand moves horizontally.

Clarify selection states

Targeting and selection

Provide feedback about what

items users can take action
on, what state those items
are currently in, and how to
take action on them.

For example, for targeting and selecting, with a gesture that requires
Z-axis movement, using size changes to show depth helps users
understand the action required (see Kinect Button, later in this
document).

Kinect for Windows | Human Interface Guidelines v1.7 67

 Feedback > Feedback Interaction Design

UI controls

For controls that are not for

navigation, such as check
boxes and toggle buttons,
have clear visual changes resting hover selected
that show state changes.

Selection confirmation

Whether the user triggers

an item by voice or gesture, Lorem ipsum
have a visual indication of
recognition before changing
the state of the application.

Kinect for Windows | Human Interface Guidelines v1.7 68

 Feedback > Feedback Interaction Design

Use progress indicators

Ergonomics

Be careful when designing Hold pose for

the timeout length for 4 seconds
progress indicators in case
users might be required to
hold a position. Also consider
how frequently users will be
repeating the interaction.
Avoid forcing users to wait
unnecessarily.

Clear visuals

If you’re using a progress

timer, or a countdown, use
clear visuals to show the
entire progression.

If you are visualizing the user’s

progress in completing a task,
make sure the visual is clear,
prominent, and placed where
the user is focused.

Kinect for Windows | Human Interface Guidelines v1.7 69

 Feedback > Feedback Interaction Design

Keep context by animating

User orientation

If the UI changes based on

something (such as distance)
that the user might have
triggered inadvertently, try
animation to show where
the content is located in the
new layout.

Layout continuity

If the user is navigating, use

animation to help him or her
understand the layout.

For example, horizontal animation can show that the user has moved
horizontally in space in your application’s layout.

Kinect for Windows | Human Interface Guidelines v1.7 70

 Feedback > Feedback Interaction Design

Use skeleton tracking Make sure users know whether the sensor sees them
feedback
User resets
Full-body skeleton tracking provides a wide
Please move left
range of new application possibilities. You can If you need to track a user
use feedback both to lead and confirm the but the sensor cannot see
them, let the user know
user’s movements.
where to stand so that it can.

If you lose track of a user in

the middle of a process, pause
the process and guide the
For more user-tracking feedback
user back to a place where
information, see Engagement, later in
you can track him or her. For example, in Kinect Sports, the game tells users if they’re
this document.
getting too close and shows them at the beginning where to stand
for optimal tracking.

The User Viewer

Another way to do this is to

show a small scene viewer
or visualization to show the
user exactly what Kinect
for Windows can see at any
given time. This can help
users understand how to stay
within the right range and,
when they’re outside of it,
why things aren’t working.
(For more information, see
The User Viewer, later in this
document.)

Kinect for Windows | Human Interface Guidelines v1.7 71

 Feedback > Feedback Interaction Design

Lead by example

Training and feedback

If you want users to copy a

specific movement or action,
you can show an avatar or
animation, either before you
expect to track the users’
movement, or even during
the movement.

If you show an animation

during the user’s movement,
visually indicate whether
or not the user is doing it
correctly.

An example of this is in Xbox Dance Central, where the onscreen

dancer moves correctly and independently of the user, but limbs get
highlighted in red if the user makes a mistake.

Kinect for Windows | Human Interface Guidelines v1.7 72

 Feedback > Feedback Interaction Design

Mirror users’ actions

Real-time movements

Make sure the movement

is in real time so the person
feels in control.

Realistic reactions

Make other objects in the

scene react correctly to
collisions with the skeleton/
avatar.

Smooth movements

Apply smoothing techniques to

skeleton-tracking data to avoid
motion and poses that are jittery
or not human-like.

Kinect for Windows | Human Interface Guidelines v1.7 73

 Feedback > Feedback Interaction Design

Use audio feedback Warning sounds

Audio can be a good way

Audio feedback can be very effective, and has
to get users’ attention if
its own set of considerations. they need to be notified of
something – for example,
audio warnings or alerts.
! Microphone is not detecting
audio, please check connection

Teaching cues

You can use sound patterns

or cues to communicate a
simple message or teach
users when to pay attention.

Kinect for Windows | Human Interface Guidelines v1.7 74

 Feedback > Feedback Interaction Design

Base audio feedback on user orientation

Instructional audio

If the user isn’t facing

the screen, or is far away, You can play the
consider using audio as a next song by
saying “next”
way to communicate to him
or her – for example, giving
available options, directions
or instructions, or alerts.

Announce selection states

Activity signals

Sounds can be a good way Click

to signal to a user that
something has changed or
has been selected.

Consider using sounds that match the action the user is taking, to
enforce his or her feeling of control and familiarity – for example,
a click noise when a button is pressed.

Kinect for Windows | Human Interface Guidelines v1.7 75

 Feedback > Feedback Interaction Design

Combine feedback Reinforced feedback

Combining visual and audio

In the physical world, we use all of our senses
inputs can make feedback
to gauge whether our actions are effective stronger.
and in order to communicate naturally with
others. Similarly, combining different types of
feedback often makes for a better experience
in a virtual world.

A great example of this is the menu in Dance Central. As the user

scrolls up and down the menu, the following changes happen to the
item he or she points to:

• It grows larger.

• It glows a brighter shade of pink.

• It gets a slider visual.

• The items surrounding it become slightly

larger and a slightly different color.

• The sound changes dynamically, in real time,

based on the position of the user’s hand.

The result seems like a very “hands-on” experience, where the user
can almost feel the effects of his or her movements.

Kinect for Windows | Human Interface Guidelines v1.7 76

 Basic Interactions

Basic Interactions
Although we leave it up to you to create
exciting and unique experiences with Kinect for
Windows, we’ve taken care of some of the basic
interactions and controls for you and included
them in the Developer Toolkit. Using our
interactions and controls saves you time and
also establishes some consistency that your users
will learn to expect as they encounter Kinect for
Windows experiences in various aspects of their
daily lives. We’ve spent a lot of time tuning and
testing this first set of interactions to give you
the best foundation possible, and we’ll add more
in our future releases.

The following sections help you set the stage for

your application, and show you how to enable
users to easily engage with it, target and select
items, and scroll or pan. We also call out some
examples that you’ll find in the Interaction
Gallery, and share some tips for interactions that
we don’t provide for you.

Kinect for Windows | Human Interface Guidelines v1.7 77

 Basic Interactions > Best Setup for Controls and Interactions

Best Setup for

Controls and
Interactions
We’ve worked hard to provide great controls
and interactions, but to make them as reliable as
possible, we recommend this setup:
• Set your application to use Near mode. For
more information, see the SDK, at:
http://msdn.microsoft.com/en-us/
library/microsoft.kinect.skeletonstream.
enabletrackinginnearrange.aspx
• Optimally your users should be standing
between 1.5m and 2m away from the sensor.
• The sensor should be placed above or below
the screen, wherever it has the least amount
of tilt and can see the most of your users’ bodies.
It helps a lot if it can see your users’ heads.
• The sensor should be centered on the screen
so that users can stand directly in front of it.
• There should not be large amounts of natural
light, or people walking in front of your users.

For information about button controls,

see Kinect Button, later in this
document.

Kinect for Windows | Human Interface Guidelines v1.7 78

 Basic Interactions > Best Setup for Controls and Interactions

Screen resolution
The Kinect for Windows controls we’ve built were designed for 1920x1080 resolution screens.
Whether you’re using these controls or creating your own, keep in mind that different screen
resolutions affect the intended size of the control. Also, because the Physical Interaction Zone
(or PHIZ, described in the following section) has a fixed size relative to the person, and doesn’t
adjust for the screen size or orientation, it might be helpful as you resize your controls to focus
on fine-tuning the ratio of control size to PHIZ size, to ensure that the experience is still reliable
and the control is still easily selectable.

The smallest button we’ve designed is 220 by 220px in 1920x1080 resolution. We’ve tested to
ensure that this button size and resolution combination makes it possible to have close to 100
percent accuracy for most users. If you have a different resolution, however, you need to resize
the button to make sure the ratio is maintained, to keep the reliability. The following chart
shows how this small button size translates for different resolutions.

Kinect Region (px) Button (px)

Width Height Width/Height

UHDTV 7680 4320 880

(W)QHD 2560 1440 294

1080p 1920 1080 220

WSXGA+ 1680 1050 214

HD+ 1600 900 184

WXGA+ 1440 900 184

WXGA 1366 768 157

720p 1280 720 147

XGA 1024 768 157

SD 720 480 98

HVGA 480 320 66

Kinect for Windows | Human Interface Guidelines v1.7 79

 Basic Interactions > Setting the Stage: the Kinect Region, the PHIZ, and the Cursor

Setting the
Stage: the Kinect
Region, the
PHIZ, and the
Cursor
The Kinect Region, the Physical Interaction
Zone (PHIZ), and the cursor are all things
that you need to get started with your Kinect
for Windows–enabled interface. You can
easily set them up so they’ll work the best
for your scenario.

Kinect for Windows | Human Interface Guidelines v1.7 80

 Basic Interactions > Setting the Stage: the Kinect Region, the PHIZ, and the Cursor

The Kinect Region Full window

In the Interaction Gallery

The Kinect Region is the area on the screen
sample, shown at right,
where Kinect for Windows–enabled interactions the Kinect Region is the
are possible in your application. That means entire window; when the
it’s also the only area where your Kinect for sample is maximized, which
Windows cursor and controls will appear. As it is by default, that’s the
you plan and design your application, consider entire screen. This method
is the least jarring and
where and how big you want this area to be,
confusing for users because
and how that will affect the user experience.
it most directly translates
to the expected behavior
of cursor areas that they’ve
experienced before.

Notes

To try out how it would feel with another setup, use a mouse to click the Restore button
at the upper-right of the Interaction Gallery sample window, change the size of the
window, and drag it to different parts of the screen. Also, notice that all controls in the
sample are mouse-enabled, but are built to be used primarily with Kinect for Windows
interactions, with mouse as more of an administrative or testing input. If part of your
application is mouse- and keyboard-enabled and part is Kinect for Windows–enabled,
consider putting the non–Kinect-enabled controls inside the Kinect Region and then
hiding them, along with the mouse cursor, when the user interacts with Kinect for
Windows. For more information, see Multiple Inputs, later in this document.

Kinect for Windows | Human Interface Guidelines v1.7 81

 Basic Interactions > Setting the Stage: the Kinect Region, the PHIZ, and the Cursor

The Physical Area

Interaction Zone The area of the PHIZ is relative

to the size and location of
the user, and which hand he
The physical area in which users move their
or she is using. It spans from
hands is known as the Physical Interaction Zone approximately the user’s head to
(or PHIZ). As the user moves his or her hand the navel and is centered slightly
within the PHIZ boundaries, the cursor moves to the side corresponding to the
within the boundaries of the Kinect Region. hand the user is actively using to
interact. Each hand has its own
The PHIZ aims to: separate PHIZ.
• Provide a consistent experience that
users can learn quickly, across all Kinect for Shape
Windows–enabled applications.
Instead of mapping directly
• Enable users to comfortably reach everything
from a flat, rectangular area
they need to interact with.
that has the same shape as
• Provide responsive tracking (low latency). the screen, we take the range
of movement of human
arms into account and use
a curved surface, which
makes moving and extending
the arm within the area
comfortable and natural.

Axis

We measure X and Y
dimensions as if the curved
surface were a rectangle. We
measure Z by arm extension,
because we’ve found that
this is not always the same
as Z space between the user
and the sensor.

Kinect for Windows | Human Interface Guidelines v1.7 82

 Basic Interactions > Setting the Stage: the Kinect Region, the PHIZ, and the Cursor

Ergonomics

The PHIZ stays the same

shape and size relative to
the user, regardless of the
aspect ratio of the Kinect
Region (for example, portrait
vs. landscape), to keep users
in a space where movement
is comfortable and not
fatiguing. This means that
users will always be able to
comfortably interact with
all parts of your application;
however, keep in mind that
the thinner and longer your
Kinect Region is, the harder
it will be for your users to
target precisely.
For example, if your screen is extremely tall and narrow, users will have
to move their hands much farther to move left and right, and moving
up and down will seem extremely fast.

Kinect for Windows | Human Interface Guidelines v1.7 83

 Basic Interactions > Setting the Stage: the Kinect Region, the PHIZ, and the Cursor

The Kinect for Pressing

Windows Cursor As a user extends his or her arm in a pressing motion, a color fill goes up and down within the
cursor to indicate how far along in the press the user is. This feature was heavily user-tested
Much like a mouse cursor, the Kinect for
and has shown to improve pressing accuracy and learnability. It also makes it easier for users to
Windows cursor provides visual cues to your
cancel before a press is made, or retarget if a press is about to happen over the wrong control.
users as they target, select, and scroll or pan by
The cursor also has a visual state that indicates when the user has completed a press.
moving their hand in the PHIZ.

Although we’ve built a lot of feedback into our

controls and we suggest you do the same for
any new ones you build, we’ve made the cursor Gripping
a strong mechanism for providing feedback. When Kinect for Windows detects that the user’s hand is in a gripped state, the cursor changes
to a gripped visual and shows a color consistent with the fully pressed state. This confirms
to the user that the grip is detected, as well as being a strong visual cue for how to make a
recognizable gripped hand.

Standard cursor and feedback graphics

Default Targeting Progress Fully pressed Gripped hand Right hand vs. left
targeting over indication state detected hand cursors
state something (color fills
that is hand as the
actionable user presses
(grip or press) further)

Kinect for Windows | Human Interface Guidelines v1.7 84

 Basic Interactions > Setting the Stage: the Kinect Region, the PHIZ, and the Cursor

The cursor moves freely within the Kinect Region only when the user’s hand is in the PHIZ; if
the user’s hand is anywhere above and to the sides of the PHIZ, the cursor sticks to the top, left,
or right of the window, and becomes semitransparent. This provides feedback for how to get
back into the area where the user can interact. When the user’s hand falls below the PHIZ, the
cursor drops off the bottom of the screen and is no longer visible, allowing the user to rest or
disengage.

Because we’ve built only one-handed interactions (see Vary one-handed and two-handed
gestures, earlier in this document), by default we show only one cursor for one user at any
given time. This cursor is determined by the hand of the primary user who first enters the PHIZ
(see the following section, Engagement). The primary user can switch hands at any time by
dropping one hand and raising the other.

Kinect for Windows | Human Interface Guidelines v1.7 85

 Basic Interactions > Engagement

Engagement
With most human-computer interactions, it’s
easy to know when users mean to interact with
the computer, because they deliberately move
the mouse, touch the keyboard, or touch the
screen. With Kinect for Windows, it’s harder
to distinguish between deliberate intent to
engage and mere natural movement in front
of the sensor. We leave the details of how to
handle this up to you, because your specific
scenario might have special requirements or
sensitivity.

This section describes the default behavior and

the User Viewer control, covers some things to
think about as you design for engagement, and
explains the solution we’ve implemented in the
Interaction Gallery sample.

Kinect for Windows | Human Interface Guidelines v1.7 86

 Basic Interactions > Engagement

Default engagement Hand states

behavior Other than “Not tracked,” they are not mutually exclusive:

The Kinect for Windows interaction model • Not tracked – No hands are being tracked.
considers the engaged user to be whoever is in • Tracked – Kinect is tracking at least one skeleton.
control of the application. • Active – Hand is tracked, and raised anywhere above the bottom of the PHIZ,
By default, the engaged user is the same as the at any reachable arm extension – for example, when the arm is reached all the way
up, out to the right, or out to the left, and anywhere in between.
primary user, and the primary user is selected
based solely on user hand states observed. • Interactive – Hand is tracked, active, and inside the PHIZ (a smaller rectangle that
These defaults can be overridden. is within the active area described above).

Primary user
Only one user can control the single cursor at any time; this person is the primary user. Default
primary user selection can be overridden by your application to fit your scenario:
• The primary user is designated when the first tracked person raises his or her hand.
• When the primary user drops his or her hand below the PHIZ (out of the active area), or
leaves the view of the sensor, he or she stops being the primary user, and another skeleton-
tracked user can raise a hand to become the new primary user.
• While the primary user has an active hand, another user cannot take over.
• Even though the sensor can see up to six people, only two of them will have tracked
skeletons. The primary user can only be one of these two fully tracked users.

Notes

It might make sense for your application to cycle through tracked skeletons, to enable
all (up to six) users to have the opportunity to be active.

Kinect for Windows | Human Interface Guidelines v1.7 87

 Basic Interactions > Engagement

The User Viewer Colors

We’ve built the User Viewer control to • Color for the primary (engaged) user
help make engagement easier for users to
understand. It’s a simple view that shows
silhouettes for any user tracked by the
sensor and allows developers to set colors
for any visible players. By default, it uses two
colors to distinguish between the primary
• Color for any other users who are detected but not tracked
user and others.

We’ve chosen some generic colors as default for this control, but you can set them to
whatever’s appropriate for your design and brand.

The User Viewer helps to answer the following engagement

questions for your users:
• Can the sensor see me?
• Is any part of my body clipped? Is my hand still in view?
• Am I the primary user in control of the experience right now?
• Is it possible for me to take over control of the application?
• Someone just walked in between me and the sensor: am I still tracked?
• Is the person behind me or next to me confusing the sensor and making my cursor jumpy?

Kinect for Windows | Human Interface Guidelines v1.7 88

 Basic Interactions > Engagement

Including the User Viewer in your application helps your users understand how to place
themselves at the right distance and orientation for the sensor to see them. It also reduces any
feeling that the application or Kinect for Windows is buggy, unresponsive, or unexplainably
jumpy, and helps users figure out what is happening and how to make their experience better.
As mentioned above, this feedback is especially important in Kinect for Windows interactions,
because users moving freely in space will quickly feel a frustrating lack of control if the system
does not behave as they expected. For more information, see Feedback, earlier in this document.

You can size and place the User Viewer anywhere in your user interface.

In the Interaction Gallery sample, we use the user viewer in two different ways:

A full-screen overlay on the

Attract view
The purpose of the Attract view is to show
users that we can see them, and entice them
to interact with the application. The large
User Viewer helps to get this message across
and serves as an interesting visualization.

Small and centered at the top

on all other screens
This visually cues users to what the sensor
sees, who the engaged user is, and how to take
over engagement. We also use the small User
Viewer as a launching point for application
messaging and user education, and replace it
with the Sensor Chooser UI if there is a Kinect
for Windows connectivity error.

Kinect for Windows | Human Interface Guidelines v1.7 89

 Basic Interactions > Engagement

Considerations The top engagement interaction challenges are to ensure that:

for designing • Users who want to engage, can.

engagement • Users who aren’t trying to engage, don’t.

• Engaging feels natural and intuitive.
It’s important that when a user is ready to
• Engaging doesn’t feel like significant additional effort.
interact with your application, they can do
so easily.

Kinect for Windows | Human Interface Guidelines v1.7 90

 Basic Interactions > Engagement

Ease of engagement Quick, easy engagement Deliberate, safe engagement

When considering how to design your If you’re not concerned about false positives, A “high barrier” solution could build on
application’s engagement strategy, consider how a “low barrier” solution might simply look top of a low barrier, looking for additional
and where you expect your users to interact for a tracked user’s hand entering the PHIZ factors in order to remove false positives. For
(being interactive), such as our default example, you might want this when data or
with it. In some scenarios, it should be harder for
behavior does. For example, this might be a consequences are more critical.
users to engage.
good method for a simple interactive sign.

Notes

With the high-barrier solution, after the low-barrier criteria are met, hinting could show
what else is required to complete engagement.

You might trigger engagement when the user:

• Faces the screen
• Raises a hand into the engagement area
• Is in front of the sensor and remains stationary for a short time
(less than 500 milliseconds)

Kinect for Windows | Human Interface Guidelines v1.7 91

 Basic Interactions > Engagement

Common false Here are some common user behaviors that can
positives for potentially be misinterpreted:
• Holding something (such as a drink or mobile phone)
engagement • Moving a hand toward the body or resting it on the body
Your challenge is to design actions that • Touching the face or hair
feel easy to the user, but also don’t risk • Resting an arm on the back of a chair
false positives.
• Yawning and extending arms
• Talking and gesturing with other people

Kinect for Windows | Human Interface Guidelines v1.7 92

 Basic Interactions > Engagement

Initial engagement Here is how the engagement flow works:

In the Interaction Gallery sample, we • The application starts in Attract view, which cycles through images,
demonstrate an example of a more enticing users to come and interact.
complicated engagement model that has a • As users who walk by are detected, they appear as gray silhouettes in
“speed bump” for initial engagement and a full-screen overlay on the Attract view.
then allows deliberate engagement handoff • When a user pauses and faces the sensor, her player mask turns
between users. purple and she is directed to raise her hand. (For more information
about player masks, see Multiple Users, later in this document.)
• Once the user raises her hand, her silhouette is replaced with a
cursor, and she is presented with a large button and instructed to
“Push Here.” Note that this also serves as user education, teaching
the press-to-select interaction. In user research, we found that this
was enough to teach people to successfully press to interact with the
rest of the application.
• After she presses the large button in the center of the screen, she is
successfully engaged and enters the Home view of the application.

Notes

This is a fairly high barrier to entry, requiring a user to perform a deliberate action in
order to engage. Another option would be to skip this step and go directly to the home
page after a user raises his or her hand into the PHIZ.

Kinect for Windows | Human Interface Guidelines v1.7 93

 Basic Interactions > Engagement

User handoff The following occurs in the event that one user decides to
After initial engagement, the Interaction relinquish control to another:
Gallery sample demonstrates how users can • If, at any time, the primary (interacting) user drops her hand below
hand off to another user. the PHIZ, a second skeleton-tracked user has a chance to take over
control and is assigned a blue color in the User Viewer.
• Messaging comes out from the small User Viewer to inform the second
tracked user that he has an opportunity to raise a hand to take over.

• If the second user raises his hand, he has temporary control of the
cursor and is prompted to press a button to take over.

• If, at any time before the second user presses, the first user raises
her hand, she is still the primary user and will remain in control of
the application, removing the ability for the second user to take over.
• If the second user completes the press, he is now the primary user,
and has control over the cursor.

• If the primary user leaves, the application will give any other tracked
user the chance to confirm engagement and remain in the current view.

Kinect for Windows | Human Interface Guidelines v1.7 94

 Basic Interactions > Engagement

This solution overrides the default engagement behavior

in the following ways:
• The primary user drops his or her hand but does not completely
disengage and other users cannot immediately take over control.
• The primary user does not immediately switch when one user drops
his or her hand and the second user raises his or her hand.
• A third User Viewer color is used for the candidate user who has the
option of taking over control.
• The candidate user who is trying to take over control is temporarily
the primary user, but will not remain so if he or she does not
complete the required speed-bump action.

Kinect for Windows | Human Interface Guidelines v1.7 95

 Basic Interactions > Targeting

Targeting
After users have engaged with a Kinect for
Windows application, one of the first tasks is
to target an object (get to an area or item they
want to open or take action on).

Where people traditionally move a mouse or

move their hand over a touch screen, they can
now move their hand in the PHIZ to control a
cursor on the screen. We’ve enabled targeting
from a distance to enable natural integration
with other distance-based interactions.

We’ve found that people naturally move

their hands in the X, Y, and Z axes even when
they think they’re moving only in a single
axis. Our ergonomic PHIZ helps translate to
the screen what users think and feel they are
doing. As with any new input method, there’s
a slight learning curve, but users quickly learn,
especially as they continue to encounter the
same experience in all Kinect for Windows–
enabled applications.

As you design interfaces where users target

objects, consider the following.

Kinect for Windows | Human Interface Guidelines v1.7 96

 Basic Interactions > Targeting

Do Don’t

Make controls easy to target, with Crowd multiple controls together or

appropriate spacing, placement, and sizes. make them too small to reliably hit.

(Our recommended smallest button size is

220px; see Screen Resolution.)

Make it clear which objects the user Make users guess which items they can
can take action on. interact with.
ACTIONABLE NON-ACTIONABLE

Lorem
ipsum dolor
Lorem
sit amet,
consectetur

Lorem ipsum dolor Lorem ipsum dolor

Make sure visual feedback matches Provide ambiguous feedback.

user intent.

Provide audio feedback when the user

targets an actionable item.

Kinect for Windows | Human Interface Guidelines v1.7 97

 Basic Interactions > Selecting

Selecting
Typically, users target an item in order to take
action on it. For example, they may want to
select a button that triggers a reaction, such as
navigation, in an application. In this document,
when we refer to selecting, we mean selecting
an item and triggering an action.

Kinect for Windows | Human Interface Guidelines v1.7 98

 Basic Interactions > Selecting

With the Kinect Button and Kinect Cursor components in the Developer Toolkit, we are
introducing a new interaction for selecting with Kinect for Windows: pressing. We’ve
found that when presented with a Kinect for Windows control, users naturally think of
pressing with their hand in space; it maps nicely to their understanding of the physical
world where they press physical buttons or extend their arm to point at objects.
Currently, the components we’ve built support only the ability to select an item to
trigger an action, but you could extend and build on the interaction to make it work
for other selection scenarios, such as toggling multiple items in a list.

Along with the press interaction, we’ve provided you with two styles of Kinect Button
as components in the Developer Toolkit, designed specifically to provide a good
pressing experience as well as feedback. We’ll cover our new button styles and tips
later in this section.

Notes

Although we’ve built only buttons so far, we think it’s a good idea to use the press
interaction for other selectable controls where it feels natural. If you’re building your own
Kinect for Windows controls, consider using the press gesture where it translates well, to
keep experiences consistent and avoid forcing users to learn a large gesture set.

Kinect for Windows | Human Interface Guidelines v1.7 99

 Basic Interactions > Selecting

Why is pressing As we designed the pressing model, our goals were:

better than hovering? • Users can select with no training.
• Users can select anywhere on the screen.
The Basic Interactions sample in the 1.6 version
of the Developer Toolkit showed hovering as • Performance improves over the hover model.
the solution for selection. In that model, a user
must target a control, and then hold his or her
hand over that control for a specified amount
of time to make the selection. Although it’s
easy to learn, and dependable, hovering
over controls is not something users can get
1.6 hover model 1.7 press model
better or faster at, and as a result, selection
interactions can seem tedious and slow. People
also tend to feel anxious about the hover
progress timer starting, so they keep moving We want users to be confident when they press to select and we want to make
their hand, or feel they have to hold their hand inadvertent selections rare. Here are some of the problems we’ve worked to solve while
in a blank area so as not to inadvertently select translating users’ hand movements and arm extensions in space to a press on the screen:
something. Pressing allows users to be precise,
• A user means to simply target (moving in X and Y axes) when
but to go at their own pace and use a more they are physically making some movements in Z.
natural movement.
• A user means to push or press “straight out” in Z when, in fact,
he or she is moving quite a lot in X and Y.
• Different users will start pressing at different arm-extension levels
(for example, some with their arm close to their body, some with
their arm almost fully extended).
• Users need to cancel an inadvertent press gesture.
• Users need visual progress feedback: how much more do they need
to press? Are they pressing when they don’t mean to?
• Users need to learn to press in order to select.

Kinect for Windows | Human Interface Guidelines v1.7 100

 Basic Interactions > Selecting

Kinect Button In designing pressing with our button controls, we decided to trigger on release, similarly
The Developer Toolkit provides you with two to standard touch and mouse behavior. This enables users to cancel a press at any time
styles of Kinect Button, a component we’ve by moving their hand up, left, or right. Moving their hand down will trigger a release,
built to make the most common selectable however, because we found that as people get confident with their presses, they often
control easy to add to your application and drop their hand as they release.
to customize for your experience. We’ve Although the buttons were designed for Kinect for Windows interactions, they also
built in feedback to help users understand respond to mouse, keyboard, and touch input.
that the control is targetable, which control
they’re targeting, and how to interact with it
in 3D space. A combination of the button’s
changing size and the cursor visuals helps
users understand that they need to press and
how far they need to press in order to select
it. It also shows them if they’re in the middle
of a press so they can cancel if they need to.

Kinect for Windows | Human Interface Guidelines v1.7 101

 Basic Interactions > Selecting

Button styles Tile buttons

We are providing two button styles you can We provide three sizes for tile buttons: small
use: tiles and circle buttons. (220px), medium (330px) and large (440px).

You can change button sizes, shapes, and

colors, and also create your own, but we think
these styles are a good baseline and will cover
most of the common scenarios. The buttons are
the recommended sizes for 1920x1080 screens.
If you’re developing for a different screen
resolution, be sure to adjust the size, to make
sure your users can both hit them accurately • Tile buttons can fit in a grid, with equal padding between tiles. Padding can vary, if
and read button text from the distance you hit areas don’t overlap. For usability, button size is more important than padding.
expect them to interact from. For a chart
• Tile buttons are composed of a background (which can be a solid color or an
of how button size translates for different image/symbol) and a text area that overlays the background on the bottom edge.
resolutions, see the chart in Best Setup for
• If text is omitted, the overlay does not appear. If an image is omitted, a solid color appears.
Controls and Interactions, earlier in this
document. • You can easily resize them to fit your design.
• Tile buttons are good for listing items, launching pages, or navigating.
In the Interaction Gallery sample, we show
an example of how to use the tile button style • Tile buttons are built to be easy for users to target and select – keep them
tailored to specific experiences and styled for a somewhat large so that they‘ll remain easy to target.
specific brand. Look at the Video Player module • The border property allows for invisible padding that is hit-targetable. This allows
to see a slightly different use of a circle button buttons without dead space between them, making it easier for users to select
(which is outlined on the following page). within a group of items. By default, tile buttons have invisible 5px borders.

In the Interaction Gallery sample, we use tile buttons for:

• Tiles on the home screen that navigate you to each of the modules
• Images that launch shadow boxes in both the horizontal scrolling view and the Article view
• The buttons for the engagement speed bumps

Kinect for Windows | Human Interface Guidelines v1.7 102

 Basic Interactions > Selecting

Circle buttons Normal Hover Pressed Disabled

These are used mostly for simple
navigation buttons (such as Back or
Home) or for settings.
PLAY PLAY PLAY PLAY

• The design is a circle surrounding a glyph of your choice; you can also add text
below or to the right of the circle.
• You can scale the size of the circle and text to fit your needs and resolution.
• You can replace the circle and glyph with an image of your choice.
• Make sure that they are easy to target and select on your resolution and screen size.
• The button and text are inside a rectangular area that is all hit-targetable – this
enables users to be less accurate and still hit the button. The area of the button is
larger than the visuals, which helps reduce clutter without making users struggle
to select the buttons.
• By default you can use them in black or white, or you can re-template them to
use a brand color of your choice.

In the Interaction Gallery sample, we use circle buttons for:

• Back buttons in various views

• The Close (X) button on shadow boxes
• The Play/Pause/Replay button on the video view

Kinect for Windows | Human Interface Guidelines v1.7 103

 Basic Interactions > Panning and Scrolling

Panning and
Scrolling
Scrolling enables users to navigate up and
down, or left and right; panning can enable
users to navigate freely in X and Y within a
canvas, like dragging a finger over a map
on a touchscreen. Experiences often allow
users to scroll and pan continuously, pixel by
pixel through content, at intervals through
a surface or page, or between consecutive
screens or views.

Kinect for Windows | Human Interface Guidelines v1.7 104

 Basic Interactions > Panning and Scrolling

With the Kinect Scroll Viewer component, we’re introducing a new direct-manipulation,
continuous panning and scrolling interaction: grip and move. Kinect for Windows can detect
the user’s hand closing into a fist, called gripping. This interaction is a solution for scrolling
through lists or panning on large canvases, but might not be the strongest interaction for
navigation between discrete pages, screens, or views. The gesture allows for a high level of
control, but can be fatiguing to do repeatedly or for large jumps. You can place content or
controls inside a Kinect Scroll Viewer to add this experience to an application.

Notes

Although we’ve built grip recognition to work specifically for scrolling or panning,
consider using it for similar interactions, such as zooming, drag and drop, or rotating.

Gripping works best if the user is no more than 2m away from the sensor.

Gripping works best if users’ wrists are easily seen. Encourage users to remove large coats
or items on their wrists before interacting by gripping.

For paging or view-changing scrolling scenarios, consider using Kinect Buttons in the
scroll viewer, or above it, to help jump users to the place they’re looking for. When there
are discrete sections, it may be faster and less frustrating to navigate straight to them,
rather than scroll to them with direct manipulation.

Kinect for Windows | Human Interface Guidelines v1.7 105

 Basic Interactions > Panning and Scrolling

Why is gripping to
scroll better than
hovering?
The Basic Interactions sample from the 1.6
version of the Developer Toolkit showed an
example of scrolling through a list of items by
targeting a large button and hovering over it,
making the canvas move at a constant pace.
1.6 hover model 1.7 grip model
Like using hovering to select, it was very easy
and reliable to use, but also frustrating and slow.
Although there are ways to make hovering to
scroll work better, such as allowing acceleration,
we’ve found that direct manipulation with grip is As we worked on this new interaction with panning and scrolling in
a fun interaction and allows users to control their mind, we had the following goals:
speed and distance more deliberately.
• Provide feedback when the user grips and releases.
• Enable users to successfully scroll short distances with precision.
• Enable users to scroll longer distances without frustration or fatigue.

Kinect for Windows | Human Interface Guidelines v1.7 106

 Basic Interactions > Panning and Scrolling

Kinect Scroll Viewer

In order to provide you with the best arena for
panning and scrolling through lists by gripping,
we’ve built the Kinect Scroll Viewer for you
to use in your applications as a component
from the Developer Toolkit. Although it was
designed for Kinect for Windows input, it also
responds to mouse, keyboard, and touch input.

DETAIL

Developer Options
• You can enable and disable scrolling in X or Y axes.
• The control allows free panning when both X and Y are enabled (imagine dragging
around a canvas).
• Use the Kinect Items Control to create a databound list of items to scroll through.

User Experience
• Users can grip anywhere within the Scroll Viewer and drag to directly manipulate
the canvas.
• Users can grip and fling to scroll longer distances, and the canvas will continue
moving while being slowed by a set amount of friction.
• The Kinect Scroll Viewer tries to correct for accidental scrolls in the wrong direction
as users repeatedly fling.
• Users can stop a moving canvas at any time by gripping or pressing on the scrolling area.
• When the end of a scrollable area is reached, it has a slight elastic bounce to provide
feedback to the user.

Kinect for Windows | Human Interface Guidelines v1.7 107

 Basic Interactions > Panning and Scrolling

Users should be able to scroll or pan by gripping any portion of the screen that actually
moves when scrolled (any part of the Kinect Scroll Viewer). When the Kinect Cursor is over the
Kinect Scroll Viewer, by default the background color of the Kinect Scroll Viewer changes to
indicate the grippable area. We found that this color change makes the interaction easier to
understand and complete. The Kinect Scroll Viewer enables users to move their gripped fist
slowly for finer control, or “fling” the content if they want to traverse a longer distance. The
fling gesture is particularly helpful when users want to reach the beginning or end of a list
quickly. Kinect for Windows ignores pressing when the user’s hand is gripped, except to stop
scrolling, or unless the user has already completed a press when a grip is detected.

The visual padding at either end of the Kinect Scroll Viewer, along with the elastic effect
during scrolling, and the bounce when the end is hit from a fling, help to indicate to the user
that they’ve reached the beginning or end of a list.

We suggest that you avoid using long scrollable lists of content in applications, because
repeatedly doing any gesture can be fatiguing and frustrating. Try to ensure that most users
can reach either end of a list with no more than two or three repetitions. Grip-and-move to
scroll or pan can be a fun and novel experience, but grip recognition while users are quickly
moving their hands is not extremely reliable, so we suggest that you reserve the Kinect Scroll
Viewer for non-critical tasks. Combining grip-and-move with other gesture interactions might
also make them both slightly less reliable.

Ergonomically, horizontal scrolling is usually easier and more comfortable for people than
vertical scrolling. Where possible, structure your user interface to allow for horizontally
scrolling content.

Also, remember that, as with any new gesture, user education is important. Many users
have never experienced a grip-and-move interaction before. Grip recognition works best
when users are deliberate about their hand positions. Sometimes half-closed hands are
misrecognized as grips. Many users figure this out quickly, but having clear messaging can
help avoid initial confusion or frustration.

Kinect for Windows | Human Interface Guidelines v1.7 108

 Basic Interactions > Panning and Scrolling

User Interface
• The color of the overlay that appears when the Kinect Cursor enters the Kinect Scroll Viewer
is configurable. By default, it is a semi-transparent gray.
• The only visible part of the Scroll Viewer is the scrollbar.
• A small panning indicator shows the users’ current location in the canvas and the amount of
canvas available. Users cannot grab the panning indicator; it is only provided as visual feedback.
The panning indicator only appears when the user is actively scrolling by gripping (not with
a mouse).
• If mouse movement is detected, the scrollbar appears, with a traditional thumb control and
two arrows. After mouse movement stops, the scrollbar times out and disappears, returning
to the panning indicator.

Kinect for Windows | Human Interface Guidelines v1.7 109

 Basic Interactions > Panning and Scrolling

Notes

Horizontal scrolling is easier ergonomically than vertical scrolling. If you have vertical
scrolling, do not design it to span the entire height of the screen.

Kinect Scroll Viewer areas that take up larger screen space are easier to scroll through.

It is less fatiguing for users if they don’t have to reach across their body to scroll.

Be sure to provide clear user education when you include grip scrolling in an interface.

In the Interaction Gallery sample, the Scroll Viewer is used:

• In the Pannable Map view, for • In the Article view, for scrolling
2D panning the text vertically

Kinect for Windows | Human Interface Guidelines v1.7 110

 Basic Interactions > Zooming (Z-Axis Panning)

Zooming (Z-Axis
Panning)
Zooming makes objects on the screen larger
or smaller, or displays more or less detail. Many
people are familiar with using a zoom control
with a mouse and keyboard, or pinching to
zoom on a touch screen. Zooming with Kinect
for Windows can be especially challenging
because it’s much harder to be precise about
distance, or start and end points, when users
aren’t directly touching the surface.

Currently, we don’t provide controls or

interactions that help with zooming, but we
recommend using our grip recognition if you
do your own implementation of zooming, to
keep it consistent with the scroll interaction.

The following suggestions are based on existing

zooming UI that people might be familiar with.

Kinect for Windows | Human Interface Guidelines v1.7 111

 Basic Interactions > Zooming (Z-Axis Panning)

Triggered zoom mode

UI that represents the hands,

or a symbolic visual, such +
as arrows, to indicate that
zooming has been initiated.
This is useful if, for example,
the user triggers zooming by
holding both hands up.

Zoom control UI

UI that can be similar to a

slider from 0 to 100 percent.
It might be useful to enable
users to grab and drag the
slider, or press and hold
Forward or Back buttons on
either side.

VUI

Voice commands like

“Zoom 100%,” “Zoom in,” or
“Zoom out,” which enable
users to jump to different
zoom increments. Avoid
forcing people to do too
much repetition of voice
commands.

Kinect for Windows | Human Interface Guidelines v1.7 112

 Basic Interactions > Zooming (Z-Axis Panning)

The goal of zooming is to manipulate an object on the screen and see the results. Here
are some ways to map user actions to zoom responses. As with anything that requires
direct manipulation, try to avoid lag, and keep the UI as responsive as possible.

Proportion of change in
two-handed zoom

Look for the user to hold up

both hands, and then base
the percent of zoom on the
percent of change in the
distance of the hands toward
or away from each other.

This might feel most intuitive and familiar to users, because it’s similar
to the touch interaction. However, it can be a challenge to implement it
well. For example, you need to define the start and end of the gesture
accurately to make the zoom level stick at the appropriate location.

Z-axis zoom

Use a literal translation

between a user’s hand and
object, by zooming with the
hand moving in Z-space.
That is, increase zoom as the
user’s hand pulls in towards
the body, and decrease as
the hand pushes out.
Z-space is typically harder to work with, so this can be a more
challenging, although intriguing, approach.

Kinect for Windows | Human Interface Guidelines v1.7 113

 Basic Interactions > Zooming (Z-Axis Panning)

Trigger + zoom gesture

Enable users to trigger

zooming mode by voice or
a distinct gesture, and then
to zoom in or out by using
another gesture.

It’s important to avoid false activations with this method, because it

could be very disorienting to trigger zooming by mistake.

A set increment triggered

by a directional gesture

Specify the same increment

for every zoomed gesture.

This can be tedious and frustrating for big zooms, but can work well
for semantic zooming.

Semantic zooming is when the zoom makes a more meaningful

content change – such as zooming out of one page in a document to
see all the pages in the document, or zooming in on a collection of
tiled items to display titles or dates.

Kinect for Windows | Human Interface Guidelines v1.7 114

 Basic Interactions > Text Entry

Text Entry
Voice and gesture aren’t strong input methods
for text entry, and you should avoid designing
for it. Keep in mind that text entry with gesture
is usually a series of targeting and selecting
actions, which can get frustrating and tiring if
the user must do it multiple times in sequence.
If you expect users to need to enter text, and
they have access to alternative inputs such as
a touchscreen, it’s best to direct them to that
input method.

Kinect for Windows | Human Interface Guidelines v1.7 115

 Basic Interactions > Text Entry

Avoiding ineffective input

Writing content by gesturing

or voice is very difficult, slow,
and unreliable. Searching or
filtering can be successful if
the user only needs to select
a few letters.

Virtual keyboard

A virtual keyboard is a
text-entry UI that people
might be familiar with. It an
allows for brief text entry by
targeting and selecting from
a list of letters.

Most targeting and selecting enhancements we’ve described for other inputs can be combined
to make text entry easier. For example, it can be useful to increase collision volume (a specified
range beyond the visual boundary within which an object responds to input) based on predicted
words, or filter letters available based on completion. As always, be sensitive to being too forceful
or presumptuous about what your user’s intent is, and leave them in control.

Kinect for Windows | Human Interface Guidelines v1.7 116

 Basic Interactions > Text Entry

Do Don’t

Enable text entry for searching, or filter Require long text entry with a gesture
through a small set where only a few that imitates a keyboard experience.
letters are required.

Enable voice text entry with a small Require voice text entry with individual
number of recognized words. letters (sounds are too similar: “B,” “P,” “V”).

Use voice for short phrases and for a Require long phrase dictation or
limited and appropriate set of tasks. conversational voice input.

Kinect for Windows | Human Interface Guidelines v1.7 117

 Additional Interactions

Additional
Interactions
In addition to the basic interactions of
targeting and selecting, scrolling, zooming,
and entering text, Kinect for Windows
enables more complex distance-dependent
interactions, as well as multiple input modes
and multiple-user scenarios.

Kinect for Windows | Human Interface Guidelines v1.7 118

 Additional Interactions > Distance-Dependent Interactions

Distance-
Dependent
Interactions
With Kinect for Windows, users no longer
need to directly touch a computer in order to
interact with it. Of course, this introduces an
interesting set of considerations for designing
interactions and interfaces for larger distances.
This section describes how you can make your
interface more usable at any distance.

Kinect for Windows | Human Interface Guidelines v1.7 119

 Additional Interactions > Distance-Dependent Interactions

Interaction ranges Out of range (over 4 meters)

Users can interact with Kinect for Windows Most Kinect for Windows
interactions aren’t feasible
from a variety of distances. These distances
at this range. Your UI should
are divided into the following categories: focus on broadly informing
Out of Range, Far Range, Near Range, and users that an interesting
Tactile Range. (Note that Near Range does interaction is available, and
not refer to the Near mode setting detailed enticing them to move closer
in Meet the Kinect for Windows Sensor, with an Attract view.
Visuals must be very large and simple, and in some cases you can use
earlier in this document.) audio to get users’ attention. You could use this range to see general
shapes – for example, to see where movement is in the room.

Far (2.0-4.0 meters)

In Far Range, the user’s full

skeleton typically is visible.
Seated mode is also available.

Gestures that are fairly coarse-grained, and short commands, work

best in this range. The UI must be large enough to be visible from
far away.

Kinect for Windows | Human Interface Guidelines v1.7 120

 Additional Interactions > Distance-Dependent Interactions

Near (0.4-2.0 meters)

In Near Range, a full

skeleton might be partially
obscured or cut off, but
this is a good distance for
seated mode.

Because the user is near to the sensor, you can have fairly detailed
visuals, longer phrases for speech, and finer gestures and depth data.
This is also a good range to work in if you plan to require object or
symbol recognition.

Kinect for Windows | Human Interface Guidelines v1.7 121

 Additional Interactions > Distance-Dependent Interactions

Tactile (0.0-0.4 meters)

This is the range at which

people could use a touch
screen; thus by definition,
they must be no further
than an arm’s length away.

Because the user is close to the screen, this range can have the
highest level of detail for visuals and controls. (Many items, small
items, and fine detail.)

If your user is in Tactile Range, mouse, keyboard, and touch might be

more appropriate input methods than voice or gesture. Neither depth
nor skeleton data is available at this distance; depth data is limited to
at least 40cm in Near Range, and at least 80cm in Far Range.

Kinect for Windows | Human Interface Guidelines v1.7 122

 Additional Interactions > Distance-Dependent Interactions

Content, text, and Graphics-to-text ratio Transitions

target sizes As distance increases, consider increasing As the UI adjusts to users’ distance, use
the graphics and size of text, and reducing transitions to help them keep their context
As the user gets further away from the screen, the amount of text to maintain user and understand how their movements affect
the text and target sizes should adjust to engagement and reduce visual strain. what they see.
account for what he or she can see, and what
Kinect for Windows can adequately hear.
Also consider the increasing imprecision and
decreasing detail in hand-tracking and gestures
over distance.

User orientation Visual consistency

Consider using anchor points or unchanging Be consistent in the general location of items
sections to orient users and provide consistency. that users can take action on.

Kinect for Windows | Human Interface Guidelines v1.7 123

 Additional Interactions > Distance-Dependent Interactions

Item placement and Object grouping Distance adjustments

quantity In general, grouping items in an easy- With longer distances, text has to be larger
to-reach area can help users have faster and gestures less detailed, which will most
Two things factor into determining the interactions. Corners and edges tend to be likely lead to fewer gestures and voice
number of items on the screen that users harder to target than the center. commands in Far Range.
can take action on, and where the items
should be placed:
• The distance between the user(s) and
the sensor
• Screen size and resolution

When designing screens, keep the following

in mind.

Kinect for Windows | Human Interface Guidelines v1.7 124

 Additional Interactions > Multiple Inputs

Multiple Inputs
Much as we often use both voice and
gesture to communicate with each other,
sometimes the most natural way to interact
with an application can involve multiple input
methods. Of course, with some options and
combinations, you risk creating a confusing
experience. This section outlines the ways to
make multimodal actions clear and usable.

We recommend designing your Kinect for

Windows controls to work with multiple
inputs to allow for greater flexibility and
easier troubleshooting. Kinect Button and
Kinect Scroll Viewer are examples of controls
that work with Kinect for Windows as well as
mouse, touch, and keyboard.

Kinect for Windows | Human Interface Guidelines v1.7 125

 Additional Interactions > Multiple Inputs

Based on your use scenario, think about how you want your controls to
handle multiple inputs:
• Will users interact with the application by using more than one input
at any given time?
• Do you want users to be able to use any input method or input at any time?
Only with certain controls? Only at specified times in your task flow?
• Does one input method take priority over the others if multiple methods
are used at once?
If the Kinect Button and Kinect Scroll Viewer detect multiple inputs, they respond to
the first one detected.

In the Interaction Gallery sample, either the mouse cursor or Kinect Cursor can be
visible and used at any given time. The Kinect Cursor is on by default, but if mouse
movement is detected:

• The Kinect Cursor is disabled and invisible.

• The mouse cursor becomes visible.
• The Mouse Scrollbar appears on the Scroll Viewer.
• The application frame with windowing controls becomes visible if the application
is in full-screen view (which it is by default).
• After the mouse has been inactive for two seconds, Kinect for Windows mode
comes back on and reverses the changes.
• All controls in the application respond to mouse, keyboard, and touch input
at any time.

Kinect for Windows | Human Interface Guidelines v1.7 126

 Additional Interactions > Multiple Inputs

Single mode User cues Backup methods

interactions Provide visual and/or audio cues to Whenever possible, have a back-up input
the user about what input methods are method that users can switch to in order to
With single-mode interactions, multiple available. complete any given task (for example, either
input modes are available but only one mouse or voice for selecting a button).
is used per action. For example, your
application might allow people to use voice
to trigger some commands, and gesture to
navigate; or it might use keyboard for text
input in Tactile Range and skeleton tracking
and gestures at a distance. Here are a few
things to remember.

Kinect for Windows | Human Interface Guidelines v1.7 127

 Additional Interactions > Multiple Inputs

Multimodal Speech + gesture

interactions
Add to
The user points to a product, cart
and then says “Add to cart.”
With multimodal interactions, the user employs
multiple input methods in sequence to
complete a single action.

Speech + touch
Send
photo
The user presses and holds a
photo, and then says “Send
photo.”

Speech + skeleton

The user says “Record,” and

then makes a movement. Record

Kinect for Windows | Human Interface Guidelines v1.7 128

 Additional Interactions > Multiple Inputs

Here are some reasons to use multimodal interactions:

Reduce the number of steps for a Enable triggering controls without

more complex action moving the user’s body
For example, sending a photo typically takes For example, if a task relies on a user’s body
two steps using the same input method – being in a certain position, voice is a great
clicking the photo and then clicking Send. way to trigger an action without the person
With multimodality, this can become one having to move.
action with two input methods.

Record
Send

Increase confidence Make the best use of each input

For example, it’s more likely that a command For example, you might require voice input
will be not be executed accidentally if the to put the application into volume-setting
user has to do two things to perform it. mode, but require gesture to give the input
for the actual volume level.

Volume

Kinect for Windows | Human Interface Guidelines v1.7 129

 Additional Interactions > Multiple Users

Multiple Users
Kinect for Windows can track up to two
skeletons and, therefore, detailed movement
information from two users simultaneously.
This opens the way for some interesting
collaborative interactions, as well as new
challenges regarding control and inputs.

Currently our Kinect for Windows controls

and interactions don’t support multiple
simultaneous users. Only one person can be
actively interacting with them at any time.

This section covers what we know so far about

designing for multiple users.

Kinect for Windows | Human Interface Guidelines v1.7 130

 Additional Interactions > Multiple Users

Tracking
Kinect for Windows can be aware of up to
six people, tracking two of them in skeleton
detail, returning full joint information. For the
other four users, Kinect for Windows returns
a small amount of information: a player mask
specifying the mask or volume of the person,
and a center position.

Your application needs to select the two people who will be primary. We recommend that
the first two skeletons detected become the primary users by default.

Kinect for Windows | Human Interface Guidelines v1.7 131

 Additional Interactions > Multiple Users

Multiple users and

distance Tactile Near Far Out of range
As you design experiences and build your
expectations, keep in mind the number
of people who can physically be in view,
Number of People 0 1-2 1-6 0
depending on their distance from the sensor.
You can expect Kinect for Windows to track
the following number of people in each
distance range.

Kinect for Windows | Human Interface Guidelines v1.7 132

 Additional Interactions > Multiple Users

Multiple users, Collaborative interactions

multimodality, and Collaborative interaction is when two users interact with the same screen space and

collaboration inputs. This means that any action taken by one user affects the state of the application
for both. There are two options for balancing control in this case:

Enabling multimodality with multiple users

means that you can have many combinations
Single-driver model Equal participation model
of people at different distances, using
different input methods. Depending on your This model assigns one of the users as the This model takes input from both users, often
model for collaboration, there are some “driver” at any given time and registers simply in the order that it’s given. This works
actions taken by that person only. The driver very well for scenarios where each user has
important considerations for the different
role can be selected or transferred in a a unique experience to drive, as in games or
ways of handling this.
number of ways, including designating the avateering (making one’s skeletal movement
first user to engage as driver, or the user affect the avatar’s movement on screen).
that is closer to the sensor. This is one way However, it can be a very complex model for
to avoid conflicting inputs. Typically, this navigating or for basic interactions.
For the Interaction Gallery example of model has visuals that show which person is
handing off between single drivers, see being tracked, and has only one cursor on
User Handoff, earlier in this document. the screen at any time.

Kinect for Windows | Human Interface Guidelines v1.7 133

 Additional Interactions > Multiple Users

Non-collaborative interactions
In a non-collaboratve interaction, each user has his or her own sub-experience within
the interface.

You can handle this with screen partitioning or sectioning. In general, this experience
should be very similar to the single-driver experience described above; however, it’s an
extra challenge to correctly map each person’s speech and actions to the corresponding
interface. Voice commands can be a challenge. Gestures and movement should have
separate visual feedback per user.

Kinect for Windows | Human Interface Guidelines v1.7 134

 Conclusion

Conclusion
Kinect for Windows is at the forefront of
the revolution known as NUI—Natural User
Interface. The next generation of human-
computer interaction, NUI lets people
interact with any device, anywhere, using the
movements and language they use every day
in their lives. Microsoft Kinect for Windows–
enabled applications open a broad range of
new possibilities for people to interact with
computers in ways that feel natural. From
business to the arts, from education to gaming,
and beyond, NUI expands the horizons of
application development.

In this document we’ve provided you with

the tenets and best practices for creating
NUI applications, and guidance on both the
basic and more advanced Kinect for Windows
interactions for gesture and voice. We hope
our suggestions help you create “magical”
experiences for your users. Your development
of touch-free, natural UI will shape the way
people experience and interact with software
applications for years to come.

Kinect for Windows | Human Interface Guidelines v1.7 135