* -*- Mode: outline -*-
* Abstract
Directional Selection is Easy as Pie Menus!
Don Hopkins
University of Maryland
Simple Simon popped a Pie Men-
u upon the screen;
With directional selection,
all is peachy keen!
Pie Menus provide a practical, intuitive, efficient way for
people to interact with computers. They run circles around
buttoned-down square old pull down menus, in both capability
and convenience.
The choices of a Pie Menu are organized in a circle around the
cursor, so that the direction of movement makes the choice,
allowing the distance to be used in other ways; essentially,
they have two outputs: direction and distance. Pie Menus
encompass many forms of input: they can utilize various types
of hardware, and their two dimensions of output can represent
many types of data.
Their circular nature makes them especially well suited for
spatially oriented tasks. Menu choices can be positioned in
mnemonic directions, with complementary items across from each
other, orthogonal pairs at right angles, and other natural
arrangements. Pie Menus can make intuitively explicit the
symmetry, balance, and opposition between choices.
Choices can be made from Pie Menus in quick, easily remembered
strokes. When the direction of a selection in a Pie Menu is
known, it can be choosen without even looking. The use of
familiar Pie Menus does not require any visual attention, as
the use of pull down menus demands.
Experiments comparing pull down menus and Pie Menus have shown
clearly that people can choose items faster and with fewer
errors from Pie Menus. They are straightforward and simple to
master, and facilitate a swift, fluent, natural style of human
computer interaction.
* Description
The choices of a Pie Menu are layed out in a circle around the
cursor, so the direction in which you move makes the choice,
allowing the distance moved to be used in other ways;
* Definition
Essentially, Pie Menus have two outputs: direction and
distance, each of which can be continuous or discrete. They
are useful for inputting various types of data, and work with
many different types of input devices.
The output of a Pie Menu is based on the offset between the
two endpoints of a path specified by some two dimensional
input device.
** Output
*** Pie Menus output direction and distance
*** Outputs can be continuous or discrete
*** Direction
**** Continuous: Dial
**** Discrete: Slice
*** Distance
**** Continuous: Pull
**** Discrete: Layer
** Input
*** Path, with two endpoints
*** Positional input (x,y)
*** Way to specify endpoints
* Advantages
Their circular nature makes them especially well suited for
spatially oriented selections. Menu choices can be positioned
in mnemonic directions, with complementary items across from
each other, orthogonal pairs at right angles, and other
natural, intuitive arrangements.
** Faster
** Easier to use
There is no need to slow the mouse down and "park" in in a
small rectangle as with conventional pull down menus.
** More reliable
** Better utilization of available information
Pull down menus require that you move in the same direction
every time you choose them (thus discarding theta), and depend
on how far you move the mouse (depending on the radius
instead).
** Take less attention to use
** Visual feedback not required
The advantage is that if the user is familiar with the menu,
no visual feedback is required to select an item. Just the
direction has to be known. If the user is familiar with the
menu, then no visual attention at all is necessary. Thus you
can be looking at something in one window while traversing
menus in another.
** Mouse ahead menu supression
** Easily learned
*** Remember directions instead of order
*** Muscle memory
** Natural way to arrange spatially related choices
** Arbitrary precision
Mouse acceleration (the magnification of mouse motion when the
mouse is moved at faster speeds) results in decreased
precision with pull down menus, but not with Pie Menus.
While it allows you to move the pointer further down the menu
with less (but faster) hand motion, there is no longer a
direct corresponence between the distance of hand and cursor
motion. Mouse acceleration does not, however, remove the
correspondence between the direction of hand and cursor motion.
Because selection from pull down menus is based on both the
direction and distance of motion, and selection from pie
menus is based just on the distance, mouse acceleration
works better with Pie Menus than with pull down menus.
In fact, mouse acceleration allows you to quickly move in the
direction you want, out away from the center of the menu,
to where you have much more precision, because of the wedge
shapes of the selection regions. The further away from the
menu center you move the pointer, the finer control you have
over the angle, thus the more control you have over the
selection. Mouse acceleration allows you to quickly and
accuratly get out to the regions of fine control.
** Choices are O(1) instead of O(n)
The time it takes to choose an item is dependend upon the
distance you have to move to choose it. With pull down menus,
the distance depends upon the position of the item in the
menu, so that adding more menu items increase the average
distance required, while requiring the same ammount of
precision to choose each item. (Precision is related to how
accuratly the mouse must be positioned in order to distinguish
a choice.) Pie menus fix the distance that must be moved to
choose an item to the size of the small "numb" circle in the
menu center, where the mouse starts out. Adding more items to
a pie menu increases the precision required to select an item,
by decreasing the area of the screen that will select that
item. Because the area is a wedge shape, extending out to the
screen edge, instead of a small rectangle, the furthur out in
the direction of the item that the mouse is moved, the more
precision there is.
** Astheticly pleasing
Pies taste great.
* Drawbacks
** Screen real estate usage
* Implementation
** menu layout
*** menu radius
*** slice positions
**** x, y offset from center (icko)
**** radius, initial angle
*** slice/layer sizes
One thing that's easy and useful to implement: when you add a
slice to a menu, you give it a size parameter. When the menu
is computed, all the slice sizes are added up, and each slice
is made 360*size/total degrees of arc in size. So if they're
all the same number, they will all be the same size. If you
give each in 100*percentage of pie, and the total is 100, it
will do the right thing. If you give them all in degrees of
arc, and the total is 360, it will do the right thing. This
was implemented in PostScript, and tested on an Apple
LaserWriter.
**** normalized relative sizes
**** bigger slices/layers are easier to choose
*** menu label
**** above menu: uwm
**** in center of menu: PostScript
*** defaults
*** inactive regions
**** numb region in center
**** inactive menu
** menu invocation
What is tracking?
The user describes a path with two endpoints with the input device
The computer translates the path into the menu output, and
provides visual feedback if necessary.
How is tracking initiated?
Specify the first endpoint
Menu centered there.
Button event
Button down
Down and up with no movement
Touching a tablet or screen
*** Pie menus ala modeless
invoked anywhere on the screen. Position can be used as
argument to function invoked with menu.
**** Bucky bits
*** Buttons, menu bars, regions on the screen
** menu display
Mouse button down events near enough to the edge of the screen
may invoke a menu, that when centered on the location of the
event, would not completely fit on the screen. It is important
to have the whole menu on the screen for two reasons: so all
the selections are visible, and more importantly, so that they
are all choosable.
The canonical solution is to force the menu to be within the
confines of the screen, by moving it by an appropriate offset.
The problem with this solution was that in moving the menu,
the center of the menu by which the selection was defined was
moved. This would leave the mouse in other than the menu
center. This violates the definition of pie menus.
The obvious solution to that was to warp the mouse to the
center of the menu if the menu had to be centered on a
different place than the mouse down event.
The problem with that was that is that mouse movement between
the time of the mouse down event, and the time when the mouse
was warped, is lost.
My initial solution to that problem was to look at the offset
between the mouse down event and the current mouse location,
and warp the mouse to the center of the menu plus that offset.
There are things to consider when there are pending mouse
events that had not already been acted upon. If, at the time
the mouse down event is received, the mouse up event that
completes the selection is already pending, then there is
already enough information to make the selection. It is not
appropriate to look at the current mouse position, because the
selection has already been completely specified. At this
point, putting the menu up serves no purpose but feedback, and
slows down interaction. The selection can be made and acted
upon without even showing a menu, if feedback is not required,
or if acting on a selection provides the feedback.
The interaction between mouse warping and pending mouse events
create weird problems. If you warp the mouse when there are
pending mouse events, the positions associated with those
events are still relative to the unwarped mouse position. The
angle towards the pending mouse up event that specifies the
current selection is different after the mouse has been
warped. Also, if the last pending event is a mouse down, the
angle between that event and the current mouse position
changes.
My solution is to only put up a menu when there are no pending
mouse button events. That makes it unnecessary to warp the
mouse if that menu would occur near the edge of the screen.
When there are events pending, they are acted upon without
putting up a menu. When there are not, and more are needed to
complete a selection, a menu is actually put up. If the menu
would not fit completely on the screen, it is moved by the
appropriate offset back onto the screen, and the mouse is
warped to its current location plus that same offset. The
mouse's current location is then tracked until the mouse up
event occurs, at which time the selection is made. During
tracking, the selection that would be made were the mouse
button be let up at the current mouse position is highlighted.
Another solution the to the screen edge problem could be to
Have the mouse wrap around to the opposite edge of the screen
when it moves off. When tracking Pie Menus, you would look at
the actual displacement that the physical mouse moved from the
center of the menu, not the position the cursor was wrapped to
on the screen.
You can only warp relative input devices. Does not work with
touch screens, etc.
*** when to display
When there are no mouse button events in the input queue that would
select from the menu. Only display when all caught up with
events.
*** where to display
If possible, center the menu on the position of mouse button
event that invoked it, not the current mouse position.
When traversing nested menus, and the selection is made upon
the mouse up, and the next menu put up centered there. Upon
the next mouse down, the mouse is warped back to the center of
the menu. Be careful how this interacts with the mouse-ahead
menu supression feature, and possible menu repositioning. Only
warp if the mouse up is not already in the queue!
**** menu repositioning
Might have to reposition the menu if it would not completely
fit on the screen. If the menu must be centered somewhere
other than the location of the mouse button event, then the
mouse should be warped by the same offset that the menu must
be moved, so that it has the same direction and distance from
the menu center as it would have had from the mouse down
event.
***** mouse warping
Never warp unless there are no pending button events. Warping
invalidates the positions associated with pending events. If
you don't display until there are no more pending mouse button
events, you won't have a problem with warping invalidating
pending event positions. You can only warp relative input
devices. Warping does not apply to touch screens, etc.
** menu browsing
Browsing only happens when there are no more mouse button
events in the input queue.
As the mouse is moved around, the item whose slice contains
the angle between the menu center and the current mouse
location should be highlighted. When the mouse up event
happens, the slice that is currently highlighted should be
chosen.
If the distance between the two endpoints is smaller than a
certain distance, then nothing is choosen from the menu.
After a mouse down has specified the first endpoint, if the
next mouse up is in about the same spot, then instead of
making a choice, keep tracking until the NEXT mouse up. If
that mouse up is also in the same place, then the menu is
cancled.
This allows the user to choose from a menu in several ways:
Mouse down and up in place, see the menu, move in a direction,
mouse down and up to choose.
Mouse down, see the menu, move in a direction, and mouse up to
choose.
Mouse down and up in place, see the menu, and mouse down and
up in the same place again to cancle the menu.
Know the menu, and mouse down and up while moving in the
direction.
*** deciding which slice the mouse is in
Gorey implementation details
Telling what slice the mouse is in: Quadrant/slope
To make mouse tracking much more efficient, when the mouse
angle changes, the current then the neighboring selections
should be compared against the new angle. No reason to start
from the top of the list of selections each and every mouse
movement. I'm thinking about efficient ways to do a fast
binary search by first looking at sector numbers, and then at
slopes when you've homed in. But if x% of the cases of mouse
movement leave you in the same sector, y% in a neighbor, and
z% somewhere else, and (pardon the ad hoc notation) x < y
<<<<<<<< z , is it worth the extra effort? The solution should
work for degenerate cases (two, one, or even zero items) to be
truly elegant.
Also, at this point, it does trig, i.e. t = atan2(x, y),
whenever you move the mouse, normalizing it such that 0 <= t <
1, and multiplying by the number of items to get the item
number. I will fix this by dividing the menu into 4 quadrants,
and calculating the boundries in terms of quadrant and slope
in that quadrant upon initializing the menu. Each item will
have a quadrant and slope field for one of its edges, and when
you move the mouse, you just figure out its quadrant and slope
by looking at the signs of x and y, and dividing. Then you
look through the linked list of selections for the one it
falls in. This eliminates all the trig except for setting up
the menu. What I have already runs quite well on a sun-2, and
roars on a Sun-3.
Another thing to think about is how to represent the angle
more efficiently. Instead of quad = {0,1,2,3} and 0 <= slope <
infinity, how about quad = {0, 1}, and -infinity < slope <
infinity ...? I don't think you could not describe straight up
and down (or wherever) though. Is there a clever way to encode
that? Or is it really worthwhile? Assuming 0 <= angle < 360
points from the menu center to the mouse, the mouse's quadrant
is (angle div 90), and the slope is tan(angle mod 90). So the
angle is 90 * quadrant + arctan(slope). The slope is more
convenient for tracking the mouse when you know its x and y
offset from the menu center. You look at the directions of
the offsets to get the quadrant, and get the slope by
dividing. The angle in degrees is useful when setting up the
menu and drawing the selections. When the menu is
initialized, the slices' angles (both boundaries, and the
center) should be recorded, as well as the slope and quadrant
calculated for the boundaries (the previous boundary should
have already been calculated for all but the first slice, so
don't recalculate it unless you just want to waste time). All
that information should be easily and inexpensivly available.
Y
^
s=-x/y | s=y/x
|
quad 1 | quad 0
x<=0,y>0 | x>0,y>=0 X
----------+---------->
quad 2 | quad 3
x<0,y<=0 | x>=0,y<0
|
s=y/x | s=-x/y
|
An even more efficient way to do quadrant/slope tests, thanks
to Vaughn Pratt: Eliminate the divide used to calculate the
cursor slope, by multiplying both sides of the slope
comparison by the denominator of the cursor slope. i.e. as
before, do the divisions to calculate the slopes of the slice
edges when laying out the menu. But when tracking the cursor,
and comparing its slope with the slice edge slopes, instead of
dividing to get the cursor slope, compare the numerator of the
cursor slope with the denominator of the cursor slope times
the slice edge slope.
**** atan2
**** Quadrant/Slope
*** give user feedback
*** highlighting
*** mouse cursor
*** other types of feedback
** menu selection
How is tracking terminated?
Specify the second endpoint
Menu output based on the angle and the distance between the two endpoint.
Button event
Button up
Button down then up
Withdrawing contact from a tablet or screen
If the distance between the two endpoints is smaller than a certain
distance, then nothing is choosen from the menu.
After a mouse down has specified the first endpoint, if the next
mouse up is in about the same spot, then instead of making a choice,
keep tracking until the NEXT mouse up. If that mouse up is also in
the same place, then the menu is cancled.
This allows the user to choose from a menu in several ways:
Mouse down and up in place, see the menu, move in a direction, mouse
down and up to choose.
Mouse down, see the menu, move in a direction, and mouse up to
choose.
Mouse down and up in place, see the menu, and mouse down and up in
the same place again to cancle the menu.
Know the menu, and mouse down and up while moving in the direction.
*** Mouse clicking: dmu -vs- dumdu
Both dumdu and dmu work with normal pull down menus, also.
Provided you're not doing things like having one of the
selections of the pulldown menu automatically selected by
having the menu appear with the cursor over it. As long as the
initial position of the menu is such that the mouse is over a
region in which a mouse up would be a no-op.
**** dmu
down-move-up
experienced users
quick selection
**** dumdu
down-up-move-down-up
novice users
makes browsing easier
select on position of last up event
**** dudu
down-up-down-up
cancel. no-op
* Experiment
** Formal statement of Hypothesis
** Rationale behind the hypopethesis
** Experiment design
** What factors are being tested
** Variables
** Constants
** Test procedures
** Data
** Results
** What results were expected? Why?
* Proposed applications
** Input metaphors
*** Pie Menus
**** Outputs discrete angles, and radius
**** Slices
***** Light switch (2)
***** Compass (8)
***** Clock hands (12/60)
**** Slices, pull
***** a several setable
**** Layered Pie Menus
*** Pie Knobs (Theta Menus)
**** Percentage, intensity
**** Relative, absolute
**** Outputs continuous angles, and radius
**** Pie knobs with arguments
**** Layered Pie Knobs
**** Radius Amplified Pie Knob
***** Radius controls magnitude of angle
**** Dynamic mouse tracking -vs- Button event tracking
*** Pie Vectors (2-d)
**** 2-d vector
**** Line segment
**** Rectangle
**** Angle, Radius
**** 2-d Translation
**** Instead of scroll bars
*** Pie Balls
**** Point on a sphere
**** 3-d Unit vector
**** Surface orientation
**** 3-d Rotation
*** Nested/Multi-stroke
**** Mouse Tracking
**** Mouse button handling
**** Highlighting
**** Walking menus
*** Pie Scroll Bars
*** Pull Down Pie Menus
*** Static Pie Menus
The menu is always displayed in a region of the screen.
Mousing anywhere in the region warps the mouse to the center
of the menu and proceeds tracking from there.
The menu is always displayed in a region of the screen.
Mousing anywhere in the region warps the mouse to the center
of the menu and proceeds tracking from there.
** Display styles
Menus can be displayed as labeled icons around the menu name
label in the center, which may have arrows, spears, wedges, or
other embelishments around it. For entries that invoke a
submenu, the icon for the entry could be an scaled down image
of the menu that it invokes (possibly with less detail) so
that you can know by looking at the first menu what the next
stroke should be. Alternativly, just the name of the submenu
and maybe an icon could be displayed, but when tracking the
mouse, selections that are menus would be highlighted by
drawing the mini-menu. This might look less cluttered. When a
submenu is selected, its mini-menu would zoom to normal size,
and the parent menu could either vanish or stay underneath.
*** Labeling
**** Text
**** Icons
**** Pointers
**** Wedges
**** Continuous
*** Tracking
**** Highlighting
***** Color
***** Reverse video
***** Outline
***** Arrow
**** Mouse cursor shape
**** Animation
**** Submenus
***** Iconic version of submenu follows cursor
***** Show full sized submenu when far enough out
**** Immediate feedback
Display the effects that the menu would have were the
current selection made.
** Input devices
*** Relative
**** Mouse
**** Track ball
**** Mole (foot mouse)
**** Left/right, up/down Atari joystick
**** Arrow keys on key pad: 8 directions
*** Absolute
**** Touch pad
***** On/off
***** Pressure sensative
**** Touch screen
**** Graphics tablet
**** Joystick
**** Lightpen
**** Eye motion sensors
*** Other
**** Buttons
***** Specify path endpoints
***** Bucky bits
**** Function keys
**** Pressure
* Philosophy
I want get as much useful information out of the mouse as
possible, in ways that are:
Practical
Intuitive
Efficient
Pie menus are practical because they're general purpose. They
can be implemented on many types of hardware, and they can be
used to input many types of data.
For a mode of interaction to be intuitive, there must be an
obvious, well known, or natural correspondance between the
visual presentation, the actions that the user can take, and
their effects.
Efficiency depends on the ammount of attention and precision
needed to pass information from the user to the machine, and
how productivly the information is used.
** Slice placement
*** Complementary and paired selections
(Illustration 1)
*** Orthogonal selections
(Illustration 1)
*** Directionaly related selections
(Illustration 1)
*** Mnemonic directions
(Illustration 1)
**** Dynamically changing menus could always have "current item" at topa
** Convenience
*** Easy directions
*** Angle size
*** Precision
** Learnability
*** Novice users
*** Expert users
*** Learning curve
** Asthetics
*** Symetry
*** Balance
*** Provides a consistant interface for inputing a wide variety of data
* Glossary
** piebald
Of different colors. Spotted or blotched with black and white.
Skewbald. Composed of incongruous parts. Heterogeneous.
** slice
** layer
** dmu
** dumdu
** dudu
** event
Mouse events are asynchronous. They cause interrupts, at which
time the pertinent states are recorded in the event, and the event
is put into a queue.
*** button event
A mouse button event specifies the position of the mouse at the
time of the event, the button involved, and whether it was up or
down. In some cases, the button specification might include other
information such as bucky bits in effect at the time of the event.
*** movement event
The current mouse position is continuously updated whenever
the mouse moves. There may be unprocessed, pending mouse
events in the queue that correspond to locations that the
mouse was at before it was at the current position.
** event queue
** bucky bits
Bucky bits describe the set of modifier keys being held down, such
as Control, Meta, Shift, Command, Function, Computer Manufacturer
Logo, etc...
Double Bucky
(Sung to the tune of "Rubber Duckie")
Double bucky, you're the one!
You make my keyboard lots of fun
Double bucky, an additional bit or two:
(Vo-vo-de-o!)
Control and Meta side by side,
Augmented ASCII, nine bits wide!
Double bucky, a half a thousand glyphs,
plus a few!
Oh,
I sure wish that I
Had a couple of
bits more!
Perhaps a
Set of pedals to
Make the number of
Bits four:
Double double bucky!
Double bucky, left and right
OR'd together, outta sight!
Double bucky, I'd like a whole word of
Double bucky, I'm happy I heard of
Double bucky, I'd like a whole
word of you!
(C) 1978 by Guy L. Steele, Jr.
(For those of you who are interested, the term "bucky bits"
comes from Niklaus Wirth, known as "bucky" to friends, who
suggested that an extra bit be added to terminal codes on 36
bit machines for use by screen editors.)
** warping
Warping the mouse changes the current position of the mouse
cursor on the screen to a new location, without any
corresponding movement of the physical mouse. It does not
change the positions associated with any events already in the
event queue at the time of the warp. Only works with relative
input devices. Does not apply to touch screen, etc.
** quadrant
** slope
** continuous
** discrete
** precision
** Fitt's law
** walking menus
** mouse acceleration
* Acknowledgments
** Mike Gallaher
** Jack Callahan
** Mark Weiser
** Ben Schneiderman
** Mitch Bradley
** Rick Furuta
** Dan Hoey
** Glenn Pierson
** Marghi Hopkins
* References