PUI Midterm
phases of Heuristic Evaluation
-1) Pre-evaluation training >Give evaluators needed domain knowledge and information on the scenario -2) Evaluation >Individuals evaluate and then aggregate results -3) Severity rating >Determine how severe each problem is (priority) >Can do this first individually and then as a group -4)Debriefing >Discuss the outcome with design team
What's a medium-fidelity prototype? When might you use it?
- Basic details and layout, minimal color - Respectable enough to show clients
Doug Engelbart
-"If ease of use was the only valid criterion, people would stick to tricycles and never try bicycles." -Team invented the mouse, 1st hyperlink implementation, groupware, interactive editing, etc.
What is a pro of Nielson's heuristic evaluations?
-1) Can be used at any phase of design and implementation >obtain feedback early in design process -2) Provide quick/inexpensive feedback to designers- good bang for your buck, also fast -3) No point in letting a user find obvious problems -4) Can use together with other usability testing methodologies
Give a UI example of preventing an error
-1) Forms that have have the field predetermined -2) Forms that tell the user exactly what type of password to input (8 chars, etc)
What is a con of Nielson's heuristic evaluations?
-1) May miss problems and find "false positives" -2) User Testing is far more accurate as it takes into account actual users and tasks -3) Requires knowledge/experience >trained experts hard to find/ expensive >should use multiple experts -4) Evaluation may identify minor, rather than major, issues
Name two (2) examples of usability metrics. How would you measure those?
-1) Time on task- efficiency; how quickly they perform tasks -2) Task completion- Completion rates -3) Whether user gets stuck or not (critical incidents) -4) Accuracy (within context of task) -5) Want to use later?- End survey? -6) Replicable/do the task consistently -7) Learnable- faster the second time -12) memorable from session to session -8) Explicable to others -9) Pleasing- high subjective satisfaction -10) Robust- minimal error rates- good feedback so user can recover -11) Out-of-box experience- first time to "oh that's cool" or being productive- see how quickly it takes someone to have a positive experience Again, note that usability goals depend on what you're trying to accomplish
Name any two roles for user studies
-1. Greeter (puts users at ease and gets demographic data) -2. Facilitator (only team member who speaks/gives instructions, encourages thinking aloud) -3. Computer (simulates feedback and response) -4. Observers (take notes and recommendations)
What's an example of a UI that does not need to be walk-up-and-use?
-Adobe Photoshop, Airplanes, Unix, Emacs, Piano "If ease of use was the only valid criterion, people would stick to tricycles and never try bicycles" - Doug Engelbart
some Common Characteristics of Good Designs
-Aesthetic / emotional connection − Allows for high performance − Learnable − Fun− Has a strong and clear value proposition − ...so, the answer is that it really depends Common Themes − Solves a problem within a set of constraints well =Useful, usable, desirable
Design Principles
-Aesthetic Design -Flow (often left-to-right) -Starting point >Where does eye go first? >Can't differentiate important from unimportant -Layout >Should fit a grid
HTML Attributes
-All HTML elements can have attributes -Attributes provide additional information about elements -Attributes are always specified in the start tag -Attributes usually come in name/value pairs like: name="value"
Heuristic Evaluation
-Cheap and fast method for finding problems -Developed by Jakob Nielsen -Basic idea: >Review a user interface >Compare each screen against a list of heuristics >See where interface does (or not) comply >Note: which heuristic it violates less important than finding a (potential) usability problem What are the heuristics? − Best practices ▪ empirical rules of thumb ▪ standards ▪ rules ▪ conventions − Tested or observed over long periods of time.
Good Design is more than Usability
-Common misconception is that all designs need to be "intuitive" or walk up and use >experts need an interface as well -"If ease of use was the only valid criterion, people would stick to tricycles and never try bicycles."− Doug Engelbart (his team invented the mouse, first hypertext implementation, groupware, interactive editing, more)
why is Good Design Important? 2
-Computing now touches every part of life >Great UX design now a differentiating factor >Great UX design also essential for reliability, security >Great UX design is only way we can achieve full potential of computing --> Just knowing what are bad designs is not sufficient to design good user interfaces
heuristic eval debriefing
-Conduct with evaluators, observers, and development team members -Discuss general characteristics of UI -Suggest potential improvements to address major usability problemsDev team rates how hard things are to fix -Make it a brainstorming session >Little criticism until end of session
Designing for Slips
-Confirmations don't always help (much) >Confirms operation more than parameters >Habituation, react to by rote -Two general strategies >1. Prevent errors >2. Improve detection and responses
What is a mistake?
-Conscious deliberation w/ wrong goal -"The division occurs at the level of the intention: A Person establishes an intention to act. If the intention is not appropriate, this is a mistake. If the action is not what was intended, this is a slip."- Don Norman, "Design of Everyday Things"
Give an example of an Organizational & Social Context constraint
-Cybersecurity constraints >US State Dept was still using MSIE 5 in late 2000s (when MSIE 8 was available!) >Password length (12+ chars, 2 symbols, etc) Legal constraints >HIPAA law means doctors can't send emails >Voting machines in 1 state had to use all caps
results of Using Heuristic Evalution
-Discount: benefit-cost ratio of 48 [Nielsen94] >cost was $10,500 for benefit of $500,000 >value of each problem ~15K (Nielsen & Landauer) >how might we calculate this value? ▪ in-house −productivity; open market −sales -Correlation between severity & finding w/ HE -Single evaluator achieves poor results\ >only finds 35% of usability problems >5 evaluators find ~ 75% of usability problems >why not more evaluators? 10? 20? ▪ adding evaluators costs more & won't find more problems
heuristic eval Severity Rating
-Estimates of need for more usability efforts − Combination of Frequency, Impact -Persistence (one time or repeating) -Should be calculated after all h-evals are in -Should be done independently by all judges 0 - don't agree that this is a usability problem 1 - cosmetic problem 2 - minor usability problem 3 - major usability problem; important to fix 4 - usability catastrophe; imperative to fix
What is Wizard of Oz with respect to UX design?
-Faking the interaction and is often used for hard to implement features >The film "The Wizard of OZ": "The man behind the curtain" >Examples:- speech and handwriting recognition- augmented reality- internet of things
Why shouldn't you ask users what they want? List at least 2 reasons.
-Gap between what people say vs. what they do and what they actually need. − Not familiar with what is possible with tech ▪ with design constraints − Constraints regarding budget, legacy code, time, etc ▪ with good design ▪ with security and privacy − Users simply don't know what they want ▪ e.g., drills vs hole ▪ e.g., anti-virus vs no data loss >Sometimes users don't know what they want (ex: remote controls, originally said "I'm not that lazy") >People are good at identifying problems they face, but not necessarily solutions
***Prototyping Philosophy***
-Have to be creative to do low fidelity prototypes -all tech can b wearable if have duct tape --> have to b creative when proto. >can proto. everything; do many iterations, have bias for action ~ex.: AR, IoT, physical
Heuristic Evaluation vs User Testing
-Heuristic Eval much faster >1-2 hours each evaluator vs. days-weeks -Heuristic Eval doesn't require interpreting user's actions -User testing is far more accurate (by def.) >Takes into account actual users and tasks >HEval may miss problems & find "false positives" -Good to alternate between HE & user testing >Find different problems >Don't waste participants
What does "Recognition over Recall" mean?
-Immediately recognize vs. of having to think back to previous interaction
Problems with the Waterfall Approach
-Implies that you design once (and get it right the first time) >Unlikely in practice (see Nintendo Wii example) -Can't easily go back up if problem downstream >Design may be unrealistic / infeasible >You're stuck, can't easily adapt if you find problems -Slow to learn what should be built >Historically, did not get feedback from users >Tend to only get feedback after building the software >Ex. "We'll fix the user interface at the end"
Difference between Wizard of Oz and Low-fi prototyping?
-Lo-fi is creating rough and ready prototypes -Wizard of oz is faking interactions to do early-stage user testing -Complementary techniques:- Can do lo-fi but no WOz- Could do WOz w/ lo-fi or even hi-fi prototypes
What's a low-fidelity prototype? What's a pro and con of low-fi?
-Low fidelity = artist's rendition with many details missing -lack of detail for unimportant things- storyboards; come from comics/filming (animation specifically) >good: can fail fast, quick, easy, cheap, low commitment, iterate quickly, anyone can understand/implement -paper proto.: goal = test w users >Problems with Lo-Fi Prototypes ~"Computer" inherently buggy ~Slow compared to real app, timing not accurate ~Hard to implement some functionality ~Animation, feedback, drag, etc− End-users can't use by themselves ~Not in context of user's work environment --> Can't show low-fi to clients: "What are we paying you for?"
Why is Good Design Important? 1
-Major part of work for systems >only ~50% implementation -Bad user interfaces cost: >Money: rework, helpdesk >Time: development effort, training costs >Reputation: e.g., brand loyalty, trust− Security: congressional file server >Lives: USS Vincennes & Flight 655
Preventing Errors
-Make it easy to do the right thing, make it harder to do the wrong thing -Have things in consistent locations − Improve "visibility" of current state of system − Don't make different controls look / feel similar >Especially if critically important -Use different codings for controls (e.g., texture, size, color, shape, operation, position)
Why is waterfall a mismatch for good UX design?
-Mismatch: expected never to go backwards , not supposed to go back "upstream," implies that you design once and get it right the first time >design may b unrealistic >slow to learn what should b built
What is a mode error? Give two examples.
-Mode errors are a particularly interesting (and dangerous) kind of slip error -Same input leads to different outputs because of the mode/state that you're in. -Devices have multiple modes and our actions are for the wrong mode. >don't see state of sys. or rarely see >user intends to do sth but can't do it bc sys. in certain mode (ex.: tryna call in airplane mode) >Examples: 1) Drive gears - accidentally in reverse & hit the gas pedal instead of drive 2) Caps lock on/off when typing in password (caps lock is a mode; can't see feedback easily) 3) Using a paint tool, drawing a lasso when wanted a rectangle (palette is a mode that you're in) 4) Trying to make a phone call while in airplane mode (doesn't do same thing in separate mode) 5) Hit "paste" but didn't focus the input field, so nothing will be pasted.
Why Prototype?
-Prototyping means building simplified versions of the system -Can get feedback on your design fast & early on− Validate assumptions quickly >Fix problems before code is written >Saves time + money by finding bugs early in the process -Can quickly experiment with alternatives -Can keep the design centered on the user >test and observe ideas with users
What is a minimum viable product (MVP)?
-Refers to the version of a product that enables a full turn of Build-Measure-Learn loop with minimum of effort and development >Cheapest, fastest thing you can build to validate assumptions
What is the waterfall method for design?
-Requirements: >Specification (what are we trying to do?) →design → coding (problem here - expect you'll get it right the first time) → integration and testing → operation and maintenance --> not supposed to go upstream
What is a slip?
-Right goal, wrong action (Trying to do the right thing, accidentally do the wrong thing.) -v common kind of error - Automatic behaviors >Ex: typos, hitting wrong menu item -Well-designed things solve for this by using feedback, or asking for confirmation before executing an action >Ex: Verify if someone wants to empty the trash bin on their computer
Iterative Design
-Short cycles of design, prototype, evaluate >Use prototypes to drive process >Progressive refinement at each iteration >Quickly validate assumptions >Fail fast >Generally, more cycles → better design -Again, no overarching grand unified theory of UX design >Instead, best practice of doing as many cycles as possible
Conducting a Heuristic Evaluation
-Small set (3-5) of evaluators examine UI > Independently check against heuristics >Evaluators aggregate findings afterward -Can perform on working UI or on sketches -At least two passes for each evaluator >First to get feel for flow and scope of system >Second to focus on specific elements -If system is walk-up-and-use or evaluators are domain experts, no assistance needed >Otherwise might supply evaluators with scenarios
Give an example of how you might use Wizard of Oz to prototype a (speech / gesture / sketch / AR) UI
-Speech UI: Have someone pretend to be the software that recognizes speech and writes it down. -AR example 1 NASA Capstone 2014 Helios: A system to help NASA engineers (on ground) and astronauts (in space) track and share information about tasks (e.g., locate tools, etc.); duct tape + paper screen = smart watch -AR Example 2 NASA Capstone 2015: Rats in Space Rodent Habitat; Help Astronauts to do procedure execution (e.g., feeding on schedule, etc.); bike helmet + phone + clamp = AR proto.
taking Context into Account
-The Navy will begin reverting destroyers back to a physical throttle and traditional helm control system in the next 18 to 24 months, after the fleet overwhelmingly said they prefer mechanical controls to touchscreen systems in the aftermath of the fatal USS John S. McCain collision. -"When you look at a screen, where do you find heading? Is it in the same place, or do you have to hunt every time you go to a different screen?"
Examples of Design Errors
-The colorfully packaged multipurpose cleaner Fabuloso has a record of mistaken identity. In 2006 researchers looked at about four months of data from the Texas Poison Center Network and found 94 cases of people accidentally ingesting the household cleaner.
General best practice is to have users read aloud a task that you hand them, why is that?
-This gets them into the mindset of think-aloud protocol
What are types & examples of slips?
-Trying to do the right thing, accidentally do the wrong thing. 1) Capture errors 2 sequences have common initial stages, alternative action 'captures' your attention. Frequently done activity takes over. - Ex: accidentally find yourself driving somewhere you're used to driving to 2) Description/similarity errors 2 objects physically alike enough to mess up. - Ex: flipping wrong light switch; throwing clothes into the garbage instead of the hamper 3) Associative activation Event activates a similar but wrong response. - Ex: office phone rings, but say "come in" 4) Data-driven errors Recalling wrong piece of data; confusing 2 numbers. - Ex: call secretary to reserve room, dial room number 5) Loss of activation Forgetting to do something, or part of the act. - Ex: go into a room and forget why 6) Mode errors Devices have multiple modes and our actions are for the wrong mode
Usability Goals
-Want to make sure we make progress in UX >Can do this by setting and measuring goals -According to the ISO: >"The effectiveness, efficiency, and satisfaction with which specified users achieve specified goals in particular environments." -This does not mean you have to create boring designs or only for novices; it all depends on your goals & users
css struct.
-What is a CSS declaration, property, selector -Selectors: element, class, ID selector →how to specify them in HTML & CSS ▪ Combining selectors with space, e.g., p.big li {} ▪ Grouping selectors with , ▪ What pseudo-class selectors are, how to use them, give one example
Pragmatic Tips for Heuristic Evaluation
-When to use Heuristic Evaluation? >All the time, all phases of design and implementation >Especially before a user study, address obvious problems w/o user -Can also teach your dev team about Heuristic Evaluation >Good bang for buck, they can find + fix problems too -Multiple degrees of depth for Heuristic Evaluation >Ex. look at UIs ~15min and point out likely problems >Ex. try UI for hour and point out likely problems
design
-design is driven by requirements (what sys. should do) >focus on core needs -a design (des. how it should b doing it) is a simplified rep. of the desired artifact >can share w team
design rep.
-site maps, storyboards, schematics, mock-up >want to make sure have progress in ux ~can do this by setting n measuring goals
block-level elements
<p>, <div>, <h1>-<h6>
inline elements
<span>, <img>, <a>
pros n cons of heuristic eval
Advantages: − Uncovers many usability problems, significantly improves a product's UX − Cheaper and faster than full-blown usability tests − Heuristics can help the evaluators focus on specific problems (i.e., lack of system feedback, poor discoverability, error prevention, etc.) -1) Can be used at any phase of design and implementation; obtain feedback early in design process -2) Provide quick/inexpensive feedback to designers- good bang for your buck, also fast -3) No point in letting a user find obvious problems -4) Can use together with other usability testing methodologies Disadvantages: − Experienced usability experts are often hard to find and may be expensive − The value of issues uncovered by evaluators is limited by their skill level − Evaluators may be unaware of any technical limitations on the design − It is based on prejudged notions of what makes "good" usability -1) May miss problems and find "false positives" -2) User Testing is far more accurate as it takes into account actual users and tasks -3) Requires knowledge/experience- trained experts hard to find/ expensive- should use multiple experts -4) Evaluation may identify minor, rather than major, issues
Why Multiple Evaluators for heuristic eval?
Every evaluator doesn't find every problemGood evaluators find both easy & hard ones
Grouping elements: why?
Grouping plays a vital role in our web page because it helps the developer to target specific classes and id which makes it easier to position, style, or manipulate the web page
Name any 3 of Nielsen's heuristics from heval (heuristic evaluations)
H2-1: Visibility of System Status H2-2: Match between System and Real World H2-3: User Control and Freedom H2-4: Consistency and Standards H2-5: Error Prevention H2-6: Recognition over Recall H2-7: Flexibility and efficiency of use H2-8: Aesthetic and Minimalist Design H2-9: Help users recognize, diagnose, and recover from errors H2-10: Help and documentation Definitions: -Visibility of System Status: system should always keep users informed about what's going on >ex.: show response time -Match between system and real world: should speak the users' language; follow real-world conventions, natural and logical >(ex: button for elevator in center of lobby rather than next to elevators) -User control and freedom: users need a way to leave unwanted states; undo and redo, make it easy to fix mistakes -Consistency and standards: follow platform conventions -Error prevention: eliminate error-prone conditions or present users with a confirmation option before committing to the action -Recognition over Recall: minimize user's memory load; make instructions easily retrievable or visible -Flexibility and efficiency of use: allow users to tailor frequent actions, shortcuts for experts -Aesthetic/minimalist design: do not contain information that is irrelevant or rarely needed; elements should be aligned and grouped, (Use a grid) -Help users recognize, diagnose, recover from errors: error messages should be expressed in plain language; constructively suggest solutions; precisely indicate the problem -Help & documentation: list concrete steps to help user with tasks, not too long, easy to search, focused on the users' task
Avoid High Fidelity Prototypes in Early Stages of Design 1
High-fi leads to wrong kind of feedback − Representation communicates "finished" ▪ Comments focus on details like color, fonts, alignment versus interaction flow, missing information, etc − "When can we have it?"
Why Good UX Design is Hard
Not part of diagram, but implicit is market forces− Internationalization − Performance needs− Multiple platforms − Compatibility − Time to develop and test ("time to market")Internal Standards: − Apple UI guidelines, Google Material design − Familiarity of product of a company
Improve detection and responses
Reduce severity of errors − Cancel and undo − Trash can for "deleting" files (fancier undo for files) Make errors more obvious − Good feedback − Uncaught or misunderstood errors tend to quickly lead to malformed mental models
Name at least 2 things you should do in user studies with respect to ethical concerns
Sometimes tests can be distressing. You have a responsibility to alleviate stress: - make voluntary with informed consent - avoid pressure to participate - let them know they can stop at any time - anonymize collected data - get IRB approval (Institutional Review Board)
Why Good UX Design is Hard (Tasks, Tech, and People)
This diagram also useful for probing why good UX design is hard Technology − Constantly changing − Building and testing software is hard People − Many considerations ▪ e.g., accessibility needs, education, typing speed, experience with mouse, distinguishing colors, etc. − Need wide range of knowledge and skills to account
Why use Lo-Fi Prototypes?
− Building high-fi prototypes takes a lot of time − However, don't want to spend weeks or months doing programming or design before feedback ▪ Build prototype →evaluate → iterate ▪ Remember design philosophy: fail fast − Alternative: Simulate prototype in early stages ▪ Sketches → evaluate → iterate --> can test multiple alternatives, almost all int can b faked − Advantages ▪ Can test multiple alternatives ▪ Almost all interactions can be faked ▪ Gets everyone on the same page ▪ Takes only a few hours ▪ Kindergarten implementation skills (e.g. shapes and lines)
Myth 4: Marketing takes care of understanding customer needs
− Does not help you understand behavior − Gap between what people say vs. what they do and what they actually need >can't really ask ppl what they want >ppl good at identifying probs they face but not soln
Myth 1: Good design is just cool graphics
− Graphics are part of bigger picture of what to communicate and how − Need to consider what people are trying to do and how to improve
Avoid High Fidelity Prototypes in Early Stages of Design 2
− High-fi prototypes take more time to make ▪ High-fidelity tools (almost all tools) encourage precision (color, alignment, fonts) ▪ Tend to waste time on small details that aren't important in early stages of design − Starting with high-fi prototypes can lead to less creativity ▪ People tend to focus narrowly on one design with high-fidelity tools− Go very deep instead of broad (compare DFS and BFS) ▪ Easy to lose track of the big picture − In early stages of design, start broadly ▪ Explore lots of different ideas first − All this mostly true for most prototyping tools
Myth 3: We can fix the interface at the end
− Overlooks cost of major changes − "A common rule we should always try to heed is to detect and fix any problem in a production process at the lowest-value stage possible"Andy Grove (Intel employee #1 and former CEO) − Also, good design includes making sure you have the right features (it's not just about usability) − Understand people's needs and addressing those --> usability is only defined in context of a user (n a task)
Myth 2: Good design is just common sense
− Why are there so many bad web sites? − So many hard to use apps? →Underestimated work needed for great UX
