* -*- Mode: outline -*- * Abstract Directional Selection is Easy as Pie Menus! Don Hopkins University of Maryland Simple Simon popped a Pie Men- u upon the screen; With directional selection, all is peachy keen! Pie Menus provide a practical, intuitive, efficient way for people to interact with computers. They run circles around buttoned-down square old pull down menus, in both capability and convenience. The choices of a Pie Menu are layed out in a circle around the cursor, so the direction in which you move makes the choice, allowing the distance moved to be used in other ways; essentially, Pie Menus have two outputs: direction and distance, each of which can be continuous or discrete. They can encompass a wide range of input styles, and can be used with various positional input devices. Their circular nature makes them especially well suited for spatially oriented tasks. Menu choices can be positioned in mnemonic directions, with complementary items across from each other, orthogonal pairs at right angles, and other natural arrangements. Pie Menus can make intuitivly explicit the symmetry, balance, and opposition between choices. Once a user is familiar with a menu, choices can be made in quick, easily remembered strokes, without even needing to see the menu. When a selection direction is known, the use of a Pie Menu does not require any visual attention, as the use of a pull down menu would demand. Experiments comparing pull down menus and Pie Menus have shown clearly that people can choose items faster and with fewer errors from Pie Menus. They are straightforward and simple to master, and facilitate a swift, fluent, natural style of human computer interaction. * Introduction * Description The choices of a Pie Menu are layed out in a circle around the cursor, so the direction in which you move makes the choice, allowing the distance moved to be used in other ways; * Definition Essentially, Pie Menus have two outputs: direction and distance, each of which can be continuous or discrete. They are useful for multifarious forms of input, and work with many different types of input devices. The output of a Pie Menu is based upon the offset between the two endpoints of a path specified by some two dimensional input device. ** Output *** Pie Menus output direction and distance *** Outputs can be continuous or discrete *** Direction **** Continuous: Dial **** Discrete: Slice *** Distance **** Continuous: Pull **** Discrete: Layer ** Input *** Path, with two endpoints *** Positional input (x,y) *** Way to specify endpoints * Advantages Their circular nature makes them especially well suited for spatially oriented selections. Menu choices can be positioned in mnemonic directions, with complementary items across from each other, orthogonal pairs at right angles, and other natural, intuitive arrangements. ** Faster ** Easier to use There is no need to slow the mouse down and "park" in in a small rectangle as with conventional pull down menus. ** More reliable ** Better utilization of available information Pull down menus require that you move in the same direction every time you choose them (thus discarding theta), and depend on how far you move the mouse (depending on the radius instead). ** Take less attention to use ** Visual feedback not required The advantage is that if the user is familiar with the menu, no visual feedback is required to select an item. Just the direction has to be known. If the user is familiar with the menu, then no visual attention at all is necessary. Thus you can be looking at something in one window while traversing menus in another. ** Mouse ahead menu supression ** Easily learned *** Remember directions instead of order *** Muscle memory ** Natural way to arrange spatially related choices ** Arbitrary precision ** Choices are O(1) instead of O(n) The time it takes to choose an item is dependend upon the distance you have to move to choose it. With pull down menus, the distance depends upon the position of the item in the menu, so that adding more menu items increase the average distance required, while requiring the same ammount of precision to choose each item. (Precision is related to how accuratly the mouse must be positioned in order to distinguish a choice.) Pie menus fix the distance that must be moved to choose an item to the size of the small "numb" circle in the menu center, where the mouse starts out. Adding more items to a pie menu increases the precision required to select an item, by decreasing the area of the screen that will select that item. Because the area is a wedge shape, extending out to the screen edge, instead of a small rectangle, the furthur out in the direction of the item that the mouse is moved, the more precision there is. ** Astheticly pleasing * Drawbacks ** Screen real estate usage * Implementation ** menu layout *** menu radius *** slice positions **** x, y offset from center (icko) **** radius, initial angle *** slice/layer sizes One thing that's easy and useful to implement: when you add a slice to a menu, you give it a size parameter. When the menu is computed, all the slice sizes are added up, and each slice is made 360*size/total degrees of arc in size. So if they're all the same number, they will all be the same size. If you give each in 100*percentage of pie, and the total is 100, it will do the right thing. If you give them all in degrees of arc, and the total is 360, it will do the right thing. **** normalized relative sizes **** bigger slices/layers are easier to choose *** menu label *** defaults *** inactive regions ** menu invocation What is tracking? The user describes a path with two endpoints with the input device The computer translates the path into the menu output, and provides visual feedback if necessary. How is tracking initiated? Specify the first endpoint Menu centered there. Button event Button down Down and up with no movement Touching a tablet or screen *** Pie menus ala modeless invoked anywhere on the screen. Position can be used as argument to function invoked with menu. **** Bucky bits *** Buttons, menu bars, regions on the screen ** menu display Mouse button down events near enough to the edge of the screen may invoke a menu, that when centered on the location of the event, would not completely fit on the screen. It is important to have the whole menu on the screen for two reasons: so all the selections are visible, and more importantly, so that they are all choosable. The canonical solution is to force the menu to be within the confines of the screen, by moving it by an appropriate offset. The problem with this solution was that in moving the menu, the center of the menu by which the selection was defined was moved. This would leave the mouse in other than the menu center. This violates the definition of pie menus. The obvious solution to that was to warp the mouse to the center of the menu if the menu had to be centered on a different place than the mouse down event. The problem with that was that is that mouse movement between the time of the mouse down event, and the time when the mouse was warped, is lost. My initial solution to that problem was to look at the offset between the mouse down event and the current mouse location, and warp the mouse to the center of the menu plus that offset. There are things to consider when there are pending mouse events that had not already been acted upon. If, at the time the mouse down event is received, the mouse up event that completes the selection is already pending, then there is already enough information to make the selection. It is not appropriate to look at the current mouse position, because the selection has already been completely specified. At this point, putting the menu up serves no purpose but feedback, and slows down interaction. The selection can be made and acted upon without even showing a menu, if feedback is not required, or if acting on a selection provides the feedback. The interaction between mouse warping and pending mouse events create weird problems. If you warp the mouse when there are pending mouse events, the positions associated with those events are still relative to the unwarped mouse position. The angle towards the pending mouse up event that specifies the current selection is different after the mouse has been warped. Also, if the last pending event is a mouse down, the angle between that event and the current mouse position changes. My solution is to only put up a menu when there are no pending mouse button events. That makes it unnecessary to warp the mouse if that menu would occur near the edge of the screen. When there are events pending, they are acted upon without putting up a menu. When there are not, and more are needed to complete a selection, a menu is actually put up. If the menu would not fit completely on the screen, it is moved by the appropriate offset back onto the screen, and the mouse is warped to its current location plus that same offset. The mouse's current location is then tracked until the mouse up event occurs, at which time the selection is made. During tracking, the selection that would be made were the mouse button be let up at the current mouse position is highlighted. Another solution the to the screen edge problem could be to Have the mouse wrap around to the opposite edge of the screen when it moves off. When tracking Pie Menus, you would look at the actual displacement that the physical mouse moved from the center of the menu, not the position the cursor was wrapped to on the screen. You can only warp relative input devices. Does not work with touch screens, etc. *** when to display When there are no mouse button events in the input queue that would select from the menu. Only display when all caught up with events. *** where to display If possible, center the menu on the position of mouse button event that invoked it, not the current mouse position. When traversing nested menus, and the selection is made upon the mouse up, and the next menu put up centered there. Upon the next mouse down, the mouse is warped back to the center of the menu. Be careful how this interacts with the mouse-ahead menu supression feature, and possible menu repositioning. Only warp if the mouse up is not already in the queue! **** menu repositioning Might have to reposition the menu if it would not completely fit on the screen. If the menu must be centered somewhere other than the location of the mouse button event, then the mouse should be warped by the same offset that the menu must be moved, so that it has the same direction and distance from the menu center as it would have had from the mouse down event. ***** mouse warping Never warp unless there are no pending button events. Warping invalidates the positions associated with pending events. If you don't display until there are no more pending mouse button events, you won't have a problem with warping invalidating pending event positions. You can only warp relative input devices. Warping does not apply to touch screens, etc. ** menu browsing Browsing only happens when there are no more mouse button events in the input queue. As the mouse is moved around, the item whose slice contains the angle between the menu center and the current mouse location should be highlighted. When the mouse up event happens, the slice that is currently highlighted should be chosen. If the distance between the two endpoints is smaller than a certain distance, then nothing is choosen from the menu. After a mouse down has specified the first endpoint, if the next mouse up is in about the same spot, then instead of making a choice, keep tracking until the NEXT mouse up. If that mouse up is also in the same place, then the menu is cancled. This allows the user to choose from a menu in several ways: Mouse down and up in place, see the menu, move in a direction, mouse down and up to choose. Mouse down, see the menu, move in a direction, and mouse up to choose. Mouse down and up in place, see the menu, and mouse down and up in the same place again to cancle the menu. Know the menu, and mouse down and up while moving in the direction. *** deciding which slice the mouse is in Gorey implementation details Telling what slice the mouse is in: Quadrant/slope To make mouse tracking much more efficient, when the mouse angle changes, the current then the neighboring selections should be compared against the new angle. No reason to start from the top of the list of selections each and every mouse movement. I'm thinking about efficient ways to do a fast binary search by first looking at sector numbers, and then at slopes when you've homed in. But if x% of the cases of mouse movement leave you in the same sector, y% in a neighbor, and z% somewhere else, and (pardon the ad hoc notation) x < y <<<<<<<< z , is it worth the extra effort? The solution should work for degenerate cases (two, one, or even zero items) to be truly elegant. Also, at this point, it does trig, i.e. t = atan2(x, y), whenever you move the mouse, normalizing it such that 0 <= t < 1, and multiplying by the number of items to get the item number. I will fix this by dividing the menu into 4 quadrants, and calculating the boundries in terms of quadrant and slope in that quadrant upon initializing the menu. Each item will have a quadrant and slope field for one of its edges, and when you move the mouse, you just figure out its quadrant and slope by looking at the signs of x and y, and dividing. Then you look through the linked list of selections for the one it falls in. This eliminates all the trig except for setting up the menu. What I have already runs quite well on a sun-2, and roars on a Sun-3. Another thing to think about is how to represent the angle more efficiently. Instead of quad = {0,1,2,3} and 0 <= slope < infinity, how about quad = {0, 1}, and -infinity < slope < infinity ...? I don't think you could not describe straight up and down (or wherever) though. Is there a clever way to encode that? Or is it really worthwhile? Assuming 0 <= angle < 360 points from the menu center to the mouse, the mouse's quadrant is (angle div 90), and the slope is tan(angle mod 90). So the angle is 90 * quadrant + arctan(slope). The slope is more convenient for tracking the mouse when you know its x and y offset >from the menu center. You look at the directions of the offsets to get the quadrant, and get the slope by dividing. The angle in degrees is useful when setting up the menu and drawing the selections. When the menu is initialized, the slices' angles (both boundaries, and the center) should be recorded, as well as the slope and quadrant calculated for the boundaries (the previous boundary should have already been calculated for all but the first slice, so don't recalculate it unless you just want to waste time). All that information should be easily and inexpensivly available. **** atan2 **** Quadrant/Slope *** give user feedback *** highlighting *** mouse cursor *** other types of feedback ** menu selection How is tracking terminated? Specify the second endpoint Menu output based on the angle and the distance between the two endpoint. Button event Button up Button down then up Withdrawing contact from a tablet or screen If the distance between the two endpoints is smaller than a certain distance, then nothing is choosen from the menu. After a mouse down has specified the first endpoint, if the next mouse up is in about the same spot, then instead of making a choice, keep tracking until the NEXT mouse up. If that mouse up is also in the same place, then the menu is cancled. This allows the user to choose from a menu in several ways: Mouse down and up in place, see the menu, move in a direction, mouse down and up to choose. Mouse down, see the menu, move in a direction, and mouse up to choose. Mouse down and up in place, see the menu, and mouse down and up in the same place again to cancle the menu. Know the menu, and mouse down and up while moving in the direction. *** Mouse clicking: dmu -vs- dumdu Both dumdu and dmu work with normal pull down menus, also. Provided you're not doing things like having one of the selections of the pulldown menu automatically selected by having the menu appear with the cursor over it. As long as the initial position of the menu is such that the mouse is over a region in which a mouse up would be a no-op. **** dmu down-move-up experienced users quick selection **** dumdu down-up-move-down-up novice users makes browsing easier select on position of last up event **** dudu down-up-down-up cancel. no-op * Experiment ** Formal statement of Hypothesis ** Rationale behind the hypopethesis ** Experiment design ** What factors are being tested ** Variables ** Constants ** Test procedures ** Data ** Results ** What results were expected? Why? * Proposed applications ** Input metaphors *** Pie Menus **** Outputs discrete angles, and radius **** Slices ***** Light switch (2) ***** Compass (8) ***** Clock hands (12/60) **** Slices, pull ***** a several setable **** Layered Pie Menus *** Pie Knobs (Theta Menus) **** Percentage, intensity **** Relative, absolute **** Outputs continuous angles, and radius **** Pie knobs with arguments **** Layered Pie Knobs **** Radius Amplified Pie Knob ***** Radius controls magnitude of angle **** Dynamic mouse tracking -vs- Button event tracking *** Pie Vectors (2-d) **** 2-d vector **** Line segment **** Rectangle **** Angle, Radius **** 2-d Translation **** Instead of scroll bars *** Pie Balls **** Point on a sphere **** 3-d Unit vector **** Surface orientation **** 3-d Rotation *** Nested/Multi-stroke **** Mouse Tracking **** Mouse button handling **** Highlighting **** Walking menus *** Pie Scroll Bars *** Pull Down Pie Menus ** Display styles Menus can be displayed as labeled icons around the menu name label in the center, which may have arrows, spears, wedges, or other embelishments around it. For entries that invoke a submenu, the icon for the entry could be an scaled down image of the menu that it invokes (possibly with less detail) so that you can know by looking at the first menu what the next stroke should be. Alternativly, just the name of the submenu and maybe an icon could be displayed, but when tracking the mouse, selections that are menus would be highlighted by drawing the mini-menu. This might look less cluttered. When a submenu is selected, its mini-menu would zoom to normal size, and the parent menu could either vanish or stay underneath. ** Input devices *** Relative **** Mouse **** Track ball **** Mole (foot mouse) **** Left/right, up/down Atari joystick **** Arrow keys on key pad: 8 directions *** Absolute **** Touch pad ***** On/off ***** Pressure sensative **** Touch screen **** Graphics tablet **** Joystick **** Lightpen **** Eye motion sensors *** Other **** Buttons ***** Specify path endpoints ***** Bucky bits **** Function keys **** Pressure * Philosophy I want get as much useful information out of the mouse as possible, in ways that are: Practical Intuitive Efficient Pie menus are practical because they're general purpose. They can be implemented on many types of hardware, and they can be used to input many types of data. For a mode of interaction to be intuitive, there must be an obvious, well known, or natural correspondance between the visual presentation, the actions that the user can take, and their effects. Efficiency depends on the ammount of attention and precision needed to pass information from the user to the machine, and how productivly the information is used. ** Slice placement *** Complementary and paired selections (Illustration 1) *** Orthogonal selections (Illustration 1) *** Directionaly related selections (Illustration 1) *** Mnemonic directions (Illustration 1) **** Dynamically changing menus could always have "current item" at topa ** Convenience *** Easy directions *** Angle size *** Precision ** Learnability *** Novice users *** Expert users *** Learning curve ** Asthetics *** Symetry *** Balance *** Provides a consistant interface for inputing a wide variety of data * Glossary ** piebald Of different colors. Spotted or blotched with black and white. Skewbald. Composed of incongruous parts. Heterogeneous. ** slice ** layer ** dmu ** dumdu ** dudu ** event Mouse events are asynchronous. They cause interrupts, at which time the pertinent states are recorded in the event, and the event is put into a queue. *** button event A mouse button event specifies the position of the mouse at the time of the event, the button involved, and whether it was up or down. In some cases, the button specification might include other information such as bucky bits in effect at the time of the event. *** movement event The current mouse position is continuously updated whenever the mouse moves. There may be unprocessed, pending mouse events in the queue that correspond to locations that the mouse was at before it was at the current position. ** event queue ** bucky bits Bucky bits describe the set of modifier keys being held down, such as Control, Meta, Shift, Command, Function, Computer Manufacturer Logo, etc... (insert "Double Bucky" song) ** warping Warping the mouse changes the current position of the mouse cursor on the screen to a new location, without any corresponding movement of the physical mouse. It does not change the positions associated with any events already in the event queue at the time of the warp. Only works with relative input devices. Does not apply to touch screen, etc. ** quadrant ** slope ** continuous ** discrete ** precision ** Fitt's law ** walking menus * Philosophy I want get as much useful information out of the mouse as possible, in ways that are: Practical Intuitive Efficient Pie menus are practical because they're general purpose. They can be implemented on many types of hardware, and they can be used to input many types of data. For a mode of interaction to be intuitive, there must be an obvious, well known, or natural correspondance between the visual presentation, the actions that the user can take, and their effects. Efficiency depends on the ammount of attention and precision needed to pass information from the user to the machine, and how productivly the information is used. ** Slice placement *** Complementary and paired selections (Illustration 1) *** Orthogonal selections (Illustration 1) *** Directionaly related selections (Illustration 1) *** Mnemonic directions (Illustration 1) **** Dynamically changing menus could always have "current item" at topa ** Convenience *** Easy directions *** Angle size *** Precision ** Learnability *** Novice users *** Expert users *** Learning curve ** Asthetics *** Symetry *** Balance *** Provides a consistant interface for inputing a wide variety of data * Glossary ** piebald Of different colors. Spotted or blotched with black and white. Skewbald. Composed of incongruous parts. Heterogeneous. ** slice ** layer ** dmu ** dumdu ** dudu ** event Mouse events are asynchronous. They cause interrupts, at which time the pertinent states are recorded in the event, and the event is put into a queue. *** button event A mouse button event specifies the position of the mouse at the time of the event, the button involved, and whether it was up or down. In some cases, the button specification might include other information such as bucky bits in effect at the time of the event. *** movement event The current mouse position is continuously updated whenever the mouse moves. There may be unprocessed, pending mouse events in the queue that correspond to locations that the mouse was at before it was at the current position. ** event queue ** bucky bits Bucky bits describe the set of modifier keys being held down, such as Control, Meta, Shift, Command, Function, Computer Manufacturer Logo, etc... (insert "Double Bucky" song) ** warping Warping the mouse changes the current position of the mouse cursor on the screen to a new location, without any corresponding movement of the physical mouse. It does not change the positions associated with any events already in the event queue at the time of the warp. Only works with relative input devices. Does not apply to touch screen, etc. ** quadrant ** slope ** continuous ** discrete ** precision ** Fitt's law ** walking menus