HCI Exam 2

internal validity

can you reproduce the experiment multiple times yourself? - same prototypes - different users - same experimental setup, conditions

what is wizard of oz prototyping?

Human operator simulates the functionality of the system behind the scenes. Don't need to build the whole system! Just have your "wizard" simulate it for the user

goals of modeling UI flow? (3)

1) identify major UI components--high level description of their role 2) interactions between these 3) two levels possible-- at conceptual design or at physical design

analyzing usability defects. 5 things you could document

1) in what part of the large system did this occur? what subsystem or component? 2) at what stage did this occur as the user tried to carry out a task? 3) at which level in the system is there a problem? 4) how was the problem revealed? 5) what can be said about the cause?

what are techniques of wizard of oz prototyping?

1) make an interactive application without much code. front-end interface is coded 2) get feedback from real users

What are the prototyping rules? (5)

1) prototypes should not be required to be complete 2) prototypes should not need to be updated 3) prototypes should be easy to change 4) prototypes should be disposable 5) in prototyping, the goal is not the artifact itself

when is it useful to use the wizard of oz prototype?

when there is an advanced technology in your system that you don't have time to build into a prototype. when you haven't determined how best to implement a feature and want to test it first

when do you evaluate?

throughout the deisgn. evaluation is a key ingredient for successful design

observer

watches and records

how can WOO be low-fi?

when you are playing the computer. when you are the wizard controlling interactions with a paper prototype while users use it

when does a prototype become high-fi?

when you take the time to put in more detail, it looks closer to the final design, and it includes fit and finish. Also, when the goal is more about refinement about a design you've chosen

what is a usability defect?

-a mismatch between what the user wants, needs, or expects and what the system provides -a breakdown in usability -an obstacle to performing a desired task -a problem in the smooth functioning of the user/computer system

5 steps for collecting data for WOO prototypes

1) Practice with friends first 2) Once comfortable, recruit users Facilitator: Provides tasks (paper) and takes notes Wizard: Operates the interface 3) User feedback can be: Think aloud (speak freely as performing tasks) Retrospective (discuss task afterwards) Heuristic Evaluation (experts are watching interaction unfold) 4) Interaction is normally video taped so designers can review later 5) Always debrief users, reveal the wizard if necessary / appropriate

What is screen design?

- before development - define and refine screens - user evaluation: both achieving tasks and navigation & other usability criteria - representation will be close to final implementation of screens and pages

what do participants do?

- performs pre-defined tasks with or without detailed instructions on how to perform them - preferably talks to herself as if alone in a room (think out loud protocol) - if they don't know the system, they need to warm up - decide what tasks matter for your evaluation (choose among: core tasks that are done frequently, tasks that are very important to users, tasks with new features or functionality, critical tasks even if done infrequently, a task you think needs validation/exploration/understanding)

controlled comparison enables casual inference . how do we control as many external variables as possible? (4)

- users placed in groups randomly - users perform experiment in exact same environment - users are given the exact same instructions, training time - any outside variable that could effect the result of the study should be equivalent for all test subjects

4 Prototyping strategies

1) paper 2) digital 3) video 4) wizard of oz

What did we learn from Jeff Hawkins' Block of Wood? What did we not learn?

Did not learn: battery life, interface and other digital interactions, sound effects and feedback Learned: form factor, living with the device 24/7

What should storyboards convey? In terms of setting, sequence, and satisfaction?

Setting--people involved, environments, and tasks being accomplished Sequence--what steps are involved, what leads someone to use the app or system, what task is being illustrated Satisfaction--what motivates people to use this system, what does it enable people to accomplish, what need does system fill

What should one think about when measuring goals?

Thinking about the falsifiability of usability requirements. if they are falsifiable, then we should be able to design user studies that prove that the prototype is sufficient

wireframe vs. paper prototyping

Wireframes are more complete, polished, and often created with computer tools. Paper prototypes are done earlier, less polished, and seem to be used more for evaluating interaction

what are site maps?

a design representation for identifying the structure and high-level content of a website. what are the pages? how are they linked?

external validity

does your experiment apply generally to other outside settings? - different users selected from a different pool - different prototypes with same general independent and dependent variables - different designers running the experiments basically means your results apply generally to experiments with the same abstract characteristics as yours

double link?

second container requires context of first container. i.e. spell checking window in a word-processor

What is interaction design?

- later in development - focuses on user work-flows - might focus on the page level - involve users in evaluation

what are the advantages of wizards? (7)

1. Faster to make / cheaper, thus more iterative prototypes possible. 2. Creating multiple variations is very easy (no code to rewrite) 3. More "real" than pure paper prototyping or mockups 4. Identifies bugs and problems with current design 5. Places user at center of development 6. Can envision challenging to build application 7. Designers learn by playing wizard

paper prototyping tips

1. Keep materials in one place! Small interface widgets tend to get lost or damaged easily. 2. Work quickly and make reusable components. 3. If something is difficult to simulate (progress indicators, right mouse menus, hyperlinks), have the user ask if it is available and then verbally describe the interaction. 4. Backgrounds (Poster Board, etc.) can be useful to contain the prototype and provide context for the user. 5. Don't be afraid to mix and match hardware and software. Maybe a physical block of wood with paper on it. 6. When appropriate, add context by including familiar operating system elements.

What are the 2 types of prototypes?

Low-fidelity and high-fidelity

What is prototyping?

The rapid creation of an approximation to a design idea for the purpose of retrieving feedback and knowledge

Wireframing tools?

balsamiq, microsoft visio, pencil project, sketch, marvel, etc.

UI flow diagram or navigation diagram

boxes represent major UI elements. each has a title and often an ID. the goals are to get a high-level overview and trace through use-cases or scenarios

What is the role of wire frame?

in development--after finding user analysis, understand requirements. after paper prototypes, as a step towards designing the final interface. keep a low-fidelity approach & reflect on what you learned about users

facilitator/interviewer

interacts with participant, explains, guides, follows-up

what is content diagram?

it is a low-fi prototype that shows organization and structure of the UI. it is from the designer's perspective

what is parallel prototyping?

making multiple prototypes in parallel. this leads to better designs. supports transfer of positive attributes across designs

single link?

next container becomes primary context of first container

evaluators

observer, facilitator/interviewer

what do you need to know to create a content diagram?

primary tasks, objects, and use-cases, scenarios, etc. Then identify containers and task objects; decide which tasks go into each container; and define links that show navigation flow

What is paper prototyping?

using paper to produce a potential interface design

independent variable

variable that is being manipulated to study an effect via a change

what are the goals for evaluation?

- could measure variables or could just look at problem - perhaps you choose to focus on efficiency or learnability - long tasks or several short tasks

What is conceptual design?

- early in development - explore high-level issues such as different conceptual models, interaction styles, user needs, usability goals - high-level representations

What are the 3 types of design do decide between when prototyping?

1) conceptual design 2) interaction design 3) screen design

13 steps for evaluation by observation (first 6)

1) select some representative users per class (client, salesperson, manager) 2) invite them to individual sessions (should last 30-90 min; 4-6 per day) 3) if system involves user's clients in the interaction, have users bring important clients or have staff pretend to be clients 4) select facilitators and note-takers 5) prepare tasks 6) prepare notebook or form for organizing notes

4 things that go into writing tasks on paper

1) state objective, final goal 2) sequence tasks that build on each other 3) be prepared for user to fail 4) have extra tasks just in case

defects can occur at difference levels. what are the 5 levels?

1) task and goal level (system doesn't provide needed functionality) 2) conceptual level (user has wrong mental model or doesn't understand system's mental model) 3) interaction style level (system-wide issues with how the general mode of interaction is defined) 4) interaction element level (some specific component in the UI has a problem) 5) physical element level (problem typing, clicking)

what are 4 reasons for doing evaluations?

1) understanding the real world (lead to a better fit with work environment) 2) does a system or prototype meet usability requirements (checking conformance to a standard) 3) engineering towards a target (x% of novice users should be able to print correctly on 1st try) 4) comparing designs (compare with competitors or among design options)

4 steps to video prototyping

1. Outline / Storyboard the Video 2. Obtain Equipment (i.e. camera, location, people) 3. Focus on MESSAGE, not on production value 4. Film it

Low-fidelity strategies? (3)

1. Storyboarding 2. Paper Prototyping 3. Video Prototyping

What are the benefits of paper prototyping?

1. Very Fast 2. Very Cheap 3. Easy to change and adapt 4. Users (and Stakeholders) can get involved and quickly alter / update the interface 5. Often this leads to a better design because they can show you what they prefer

dependent variable

variable that is measured for change after IV is altered. time to complete a task. emotion

What is a wire frame?

a low fidelity representation of a single page or screen. it should show all content or features needed & relative sizes and locations. It should not include colors or fonts. the goal is to meet the purpose of that screen

evaluation session

a meeting where participants participated in testing

participant

a person or use who provides input and feedback to those evaluating a UI

what is as high-fi prototype?

a prototype that mimics the design very closely, but remains functionally incomplete. this might be actual code, or actual physical devices that don't work fully

leading question

a question that suggests the answer the examiner is looking for or contains the information the examiner is looking to have confirmed

usability testing / usability evaluation

a systematic testing of an interface

what are containers in content diagrams?

an abstract representation of some part of a user's work activity. includes functions required to do that activity. main container should record

"Known unknowns"

aspect of a design in which you know what you don't understand and wish to learn i.e. which color scheme is most user friendly?

What are the 3 main questions to ask in the prototyping process?

1) What are your goals for the prototype? What do you wish to learn? 2) How can you measure whether of not that goal has been achieved? How can you measure which of multiple prototypes is superior? 3) What is the minimum amount of work necessary to produce, measure, and learn from your prototype?

what can you do for each type of evaluation?

1) controlled setting--observe, ask users, and test 2) natural settings--observe, ask users, and ask experts 3) without users-- ask experts and modeling

where are three places where user evaluation can happen?

1) controlled settings involving users--usability testings and experiments in labs 2) natural settings involving users--field studies and in the wild studies to see how the product is used in the real world 3) settings not involving users--to predict, analyze and model aspects of the interface analystics

4 ways the defect can be revealed

1) detected by the system (easy to see and the system catches the invalid use) 2) detected by the user (can't find the command; don't know what to do next) 3) undetected defects (user doesn't see the problem, at least not right away) 4) inefficiencies (complex, long operations; too much time needed to think)

what are 3 ways to get around the "please the experimenter bias"?

1) double-blind studies- both the user and facilitator don't know which experimental group the user is in 2) don't let the user know what you are measuring / what you care about until the study is over 3) ask questions that cancel each other out- ask about how useful the interface was and how frustrating it is

iterative design and evaluation is a continuous process that examines what 4 things?

1) early ideas for conceptual model 2) early prototypes of the new system 3) later, more complete prototypes 4) existing systems; designers need to check that they understand users' requirements`

what are the 4 types of evaluation?

1) formative evaluation (done at different stage of development; check the product meets users' needs) 2) summative evaluation (assess the quality of a finished product) 3) diagnostic evaluation (find problems) 4) measurement evaluation (assess performance)

Why do we prototype? (4)

1) gain insight into user behavior 2) communicate ideas to other team members and stakeholders 3) collect data for arguing the best design 4) prototypes encourage reflection

What are 3 benefits of storyboarding?

1) holistic focus--helps emphasize how an interface accomplishes a task 2) avoids commitment to a particular user interface 3) helps get all stakeholders on the same page in terms of the goal

what are 5 possible problems related to experimental design?

1) reliability- does the method produce the same results on separate occasions 2) validity- does the method measure what it is intended to measure? 3) ecological validity- does the environment of the evaluation distort the results (Hawthorne Effect) 4) Biases- are there biases that distort the results 5) scope- how generalizable are the results

when did things go wrong? consider the user's goal/decide/execute cycle (4 ?s)

1) the user now wants to achieve some goal (perhaps she didn't know what to do or chose the wrong goal) 2) she decides what to do with the system to achieve. she decides upon a particular action (perhaps she can't figure out how to do what she wants or perhaps she choose the wrong action) 3) she carries out the action and the system does something in response (perhaps she cannot do the action in this interface) 4) she interprets the results and responses (perhaps she can't interpret it or misinterprets it)

4 techniques for evaluation

1) user observation (observe participants interacting with system) 2) inspection/walkthrough (no participants usually involved) 3) questionnaires, surveys 4) observation of real users in their real world

evaluation by observation. what is the importance of video?

1) without it, you see what you want to see. you interpret based on your mental model 2) in the heat of the moment, you miss things 3) minor details captured 4) you can repeatedly analyze, looking for different problems

What are the benefits of video prototyping? (4)

1. Cheap and fast 2. Great communication tools - Helps achieve common ground, ideally portable and self-explanatory 3. Can serve as a 'spec' for developers 4. Ties interface designs to tasks - aligns and orients interface choices, makes sure you have a complete interface

5 steps to making a WOO prototype

1. Map out scenarios and application flow - Enumerate ALL scenarios if possible, if not, provide guidelines in as specific a format as possible 2. Put together interface "skeletons" 3. Develop "hooks" for wizard input - If paper, hooks will be very manual, if done in software, some code will need to be developed 4. Put it all together - Where and how will the wizard provide input, how will the wizard receive input from the user 5. Rehearse wizard role with a colleague

Cost of change over time in regards to prototyping

1. Prototypes are MORE expensive to change the more high-fidelity they become 2. The most common estimate is that it's 100 times cheaper to make a change before any code has been written than it is to wait until after the implementation is complete [Jakob Nielsen]. 3. Twenty years of usability engineering experience uniformly indicates that the biggest improvements in user experience come from gathering usability data as early as possible in a design project [Jakob Nielsen].

what are the disadvantages of wizards? (5)

1. Simulations may represent otherwise imperfect (or impossible) tech 2. Wizards require training and can be inconsistent 3. Playing the wizard can be exhausting 4. Some features are difficult (or impossible) to simulate perfectly 5. May be inappropriate in some venues.

What do you put in a video prototype?

1. The WHOLE task including motivation and success - Establishing shots and narrative help 2. Draw on tasks you observe 3. Illustrate important tasks your system enables - Not in video, probably not necessary for version one of the system 4. Can help scope a minimum viable product 5. Changes what design teams argue about (in a good way)

13 steps for evaluation by observation (last 4)

10) start user verbalizing as they perform each task 11) observer watch and take notes 12) post-mortem, debriefing (express appreciation, answer any questions the participant might have) 13) analysis of results by evaluation team

13 steps for evaluation by observation (next 3)

7) set up and test experiment (hardware on which to run system, audio or video recorder, software logs) 8) do a dry run (pilot study) 9) at the Start of an Observation Session (facilitator explains the nature of the project, anticipated user contributions, why user's views are important, focus on evaluating the user interface. participants are asked to sign informed consent form)

What is the prototyping process about?

It is about defining questions regarding your designs, and building something that answers those questions

What is storyboarding? What is it about? What is it not about?

It is low-fidelity and all about tasks. It is about the tasks users want to complete and the flow of the user's interaction. It is not about pretty pictures (in fact bad drawing helps focus on task and not on imagery) or showing specific interface elements.

"Unknown unknowns"

aspects of a design that you don't know are open issues i.e. why can't old people use tiny screens

What is video prototyping?

can be any level of fidelity - can make videos with paper prototype examples, Fancy high fidelity videos are ok too. Usually, dependent on where you are in the design process

baserates

how often does 'Y' occur in the current setting? - very reasonable for some of your projects, if there is a competing product that exists

what are links in content diagram?

how the user will navigate in the UI between these functional areas.