From: mark@markssun.cs.umd.edu (Mark Weiser) Subject: pie menus experiments Date: 15 October 1986 at 04:28:14 CET To: callahan@mimsy.umd.edu, don@mimsy.umd.edu Jack, I am glad you decided to do a pi menu experiment. I did not read your experimental protocol carefully, but I think it is basically good. I will read it again tomorrow. Experiments are necessary to establish the efficacy of pi menus. If you get your data in time we, you, Don, and I, will have a super pi menu paper. -mark From: Jack Callahan Subject: progress? Date: 5 November 1986 at 23:13:14 CET To: don@tove.umd.edu Cc: mark@tove.umd.edu How are things going? I have subjects ready to run as of this Friday/next Monday. Shneiderman wants to stick with fixed menu lengths. He says that variable lengths adds to experiment complexity by adding dimensions on the current design. I agree. We have settled on 8 items. Here are my 5 sample pie-menus: North,South,East,West,NorthWest,SouthEast,NorthEast,SouthWest Open,Close,Up,Down,Black,White,Left,Right Full,Empty,Dark,Light,In,Out,Active,InActive Young,Old,Fat,Thin,Tall,Short,Top,Bottom More,Less,Larger,Smaller,Quick,Slow,Low,High Here are my 5 sample linear menus: 1,2,3,4,5,6,7,8 One,Two,Three,Four,Five,Six,Seven,Eight First,Second,Third,Fourth,Fifth,Sixth,Seventh,Eighth A,B,C,D,E,F,G,H Ape,Bear,Cat,Dragon,Elephant,Fish,Giraffe,Horse and finally, five miscellaneous menus that fall into neither category: Print,Move,Copy,Delete,Undo,Again,Find,Property Clear,Next,Skip,Center,Bold,Case,Italic,Font Apples,Spinach,Oranges,Pears,Beets,Tomato,Peach,Cabbage Deliver,Another,Destroy,Reset,Kill,Abort,Start,Change Create,Stop,Expand,Define,Shift,Lock,Search,Quit These are enough for the pilot study. In the proposal, I stated there would be 5-10 tasks (menus) per cell. We have met the minimum, let's see what the pilot results say. -- jack From: callahan@tumtum.cs.umd.edu (Jack Callahan) Subject: experiment subjects Date: 17 November 1986 at 16:02:04 CET To: don@mimsy.umd.edu I am going to post the sign-up sheets in psychology later this week and have people in for 1/2 hour sessions starting Monday (Nov 24). Later this week, we need to run pilot subjects for the draft report data. It's OK if things don't work perfectly - as long as we can run a competent experiment. I got the NeWS-makers charter message. -- jack Can we run the pilot subjects this Wed or Friday? I'm busy Thurs with classes. When the real people come, it'll be Mon, Wed, and Fri for the Thanksgiving week and the first two weeks of Dec. When is the last day of class? You won't need to be= there for the sessions unless you want to since this is my job. From: Jack Callahan Subject: the experiment begins! Date: 19 November 1986 at 21:59:52 CET To: don@tove.umd.edu, mark@tove.umd.edu I just posted the signups in psychology today and hordes (literally) of freshman psych types gobbled it up. I didn't see any actually sign-up, but there were about 20 of them standing around when I stapled it to the bulletin board. They officially start Monday morning at 9 am and run until 5 pm MWF until the last day of classes. This does not mean that I am busy MWF until school ends. This was to allow the maximum people time to sign-up at their convenience. If we overbook, I will call people and tell them we are full and take down the sign-up. We only need about 25 people which includes estimates for data points we may have to throw out. -- jack From: Jack Callahan Subject: more menus Date: 21 November 1986 at 04:53:28 CET To: don@tove.umd.edu Cc: mark@tove.umd.edu, callahan@tove.umd.edu The following is the list of menus with the goals of each trial in parens. Each menu will be seen twice - once in pie format and once in linear format. Place the menu items appropriately where necessary (like up, down belong at the top and bottom of a pie menu and placed together in a linear menu). Opposites should be placed together in a linear menu and linear tasks should be placed in clockwise order in a pie menu. Goal doesn't matter on demo and practice menus. Here are the demo menus: Using linear: Rook, Knight, Bishop, Queen, King, Bishop, Knight, Rook Using pie: Hearts, Spades, Clubs, Diamonds, Ace, King, Jack, Queen Here are the practice menus: Using linear: Home, First Base, Second Base, Third Base, Left Field, Right Field, Foul, Foul Using pie: Today, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday Here are the pie tasks: North,South,East,West,NorthWest,SouthEast,NorthEast,SouthWest (North, SouthWest) Open,Close,Up,Down,Black,White,Left,Right (Right, Down) Full,Empty,Dark,Light,In,Out,Active,InActive (Empty, Active) Young,Old,Fat,Thin,Tall,Short,Top,Bottom (Tall, Bottom) More,Less,Larger,Smaller,Quick,Slow,Low,High (High, Low) Here are the linear tasks 1,2,3,4,5,6,7,8 (4,8) One,Two,Three,Four,Five,Six,Seven,Eight (Five, Seven) First,Second,Third,Fourth,Fifth,Sixth,Seventh,Eighth (Sixth, Second) A,B,C,D,E,F,G,H (F,B) Ape,Bear,Cat,Dragon,Elephant,Fish,Giraffe,Horse (Giraffe,Dragon) and finally, five miscellaneous menus that fall into neither category: Print,Move,Copy,Delete,Undo,Again,Find,Property (Again,Move) Clear,Next,Skip,Center,Bold,Case,Italic,Font (Font,Center) Apples,Spinach,Oranges,Pears,Beets,Tomato,Peach,Cabbage (Spinach,Peach) Deliver,Another,Destroy,Reset,Kill,Abort,Start,Change (Deliver,Change) Create,Stop,Expand,Define,Shift,Lock,Search,Quit (Define,Quit) This is the data we need from each trial: SubjectNumber, MenuType, TaskType, TimeToItem MenuType = linear, pie TaskType = pie, linear, mixed TimeToItem = time from menu up to item selected. If incorrect item selected, repeat menu and accumulate time -- jack From: callahan@mome.cs.umd.edu (Jack Callahan) Subject: data files for experiment subjects Date: 22 November 1986 at 19:12:02 CET To: marks-people@mimsy.umd.edu The event traces from the experiment are going to be semi-large files that I plan to store in the directory mimsy:/tmp/callahan/tests. The files themselves will be named subjectX where X is the subject number (how original!). The reason I am using /tmp is to avoid any of these nasty OVER DISC quota problems in the middle of an experiment. My only worry is the volatile nature of /tmp. Can I expect problems? -- jack https://donhopkins.com/home/piepaper/callahan/ From: Jack Callahan Subject: results Date: 1 December 1986 at 14:29:28 CET To: mark@tove.umd.edu, don@tove.umd.edu Here is the first of two messages in a correspondence with Linda Weldon I should have cc'ed you two... ---------------------------------------------------------------- Sender: Jack Callahan:Computer Science:UofMaryland Date: 30 Nov 86 19:49:42 EST (Sunday) Subject: Re: help.... From: callahan To: weldon%tove.umd.edu, ben%mimsy cc: callahan In-Reply-to: <8611302323.AA11496@tove.umd.edu> Tell you what - I'll wait until Thursday and prime you with some stuff over E-mail. Here goes: If you remember my study, it is the pie vs pull down menus. The pie menus are the circularly formatted menus as opposed to a linear top-bottom design. The experiment is a 2x3: pie linear other tasks tasks tasks using pie menus using linear menus A sample pie task would be like compass points where a linear task would be the numbers from 1-8. All menus have the same number of items (8). Each participant does 5 menus per cell. The menus repeat in each column so that they see the same menu twice using each menu type. Thus, there are a total of 15 separate menus and the whole experiment brings them thru 30 menus. They see each cell in random order, but see each menu in each cell in the same order. Did you follow that? I ran the ANOVA and got the following. The dependent variable is SEEKTIME, the time they spent in microseconds to find the item once presented with the menu. for 300 observations (I've only run 10 subjects thus far) SOURCE F VALUE PR > F MENUTYPE 11.08 0.0010 TASKTYPE 0.26 0.7682 MEANS MENUTYPE N SEEKTIME PIE 150 2.3485 LINEAR 150 2.9968 TASKTYPE N SEEKTIME PIE TASKS 100 2.5730 LINEAR TASKS 100 2.7148 OTHER 100 2.7302 I have yet to include the position of the menu item in the analysis nor the questionnaire data. Funny, but it seems they are faster using pie menus but overwhelmingly the subjective questionnaire says they prefer linear menus. (???) -- jack p.s. I cc'ed Dr. Shneiderman because I know he's interested in this discussion. I had asked Linda for stat help in the message previous to this one. ---------------------------------------------------------------- Received: by tove.umd.edu (5.9/4.7) id AA12459; Sun, 30 Nov 86 22:31:20 EST Date: Sun, 30 Nov 86 22:31:20 EST From: Linda J. Weldon Sender: weldon@tove.umd.edu Message-Id: <8612010331.AA12459@tove.umd.edu> To: callahan@tove.umd.edu Subject: fast pie it certainly looks as though you found something! Namely, pie is faster than linear regardless of task type (and that task type doesn't make a difference.) the subjective questionnaire results are interesting -- it is sometimes the case that people prefer things that they are not as good at. What are some possible explanations? familiarity with linear menus, maybe. . . be seeing you, linda From: Jack Callahan Subject: pie menus con't Date: 9 December 1986 at 03:57:46 CET To: don@tove.umd.edu, mark@tove.umd.edu I'm working on the draft paper for Ben's class tommorrow. So far, pies are doing well. Reduced errors and faster item seek times are consistent even with more subjects. Task type has finally show statistical significance as suspected. If I remember, I'll leave copies for both of you where you can find them (don - on your desk, mark - mailbox). -- jack From: Mark Weiser Subject: pie paper Date: 19 December 1986 at 04:52:43 CET To: callahan@mimsy.umd.edu, don@mimsy.umd.edu Ok, time to write. Jack to you have your paper Ben on line somewhere? Don, are you storing notes somewhere? I can put together a draft this week, and have you guys look it over over break. - mark From: Mark Weiser Subject: Re: papers Date: 22 December 1986 at 04:00:55 CET To: don@brillig.umd.edu Cc: despina@mimsy.umd.edu Don, Diane will arrange to reimburse you for attending the Usenix, via my foundation funds. I think that we should merge relevant parts of yours and Jack's papers into a CACM paper. That will get the widest audience, and it is importnat enough. I will do the merging. It should happen fast because of the long lead time of that publication. Meanwhile, your part without the experiment could be submitted to Usenix, and Jack could try to submit just his piece somewhere (although I think his alone does not stand up as well as yours alone). -mark