Research Methods Midterm

What is a likert scale

7-point agreement scale (Strongly, Moderately,Slightly) with a neutral point• Likert scale statement must be fairly neutral: "Using this productwas easy" and not "Using this product was best eve

What is the system usability scale

en questions that relate to usability• Most widely regarded and used!• Useful for software, apps, websites, products, etc.• Evaluation of overall score• > 70: Acceptable• 50 - 70: Marginal• < 50: Not acceptable

What is the Waterfall method of product design

user-centered design)• Specify > design > build > test• Iterative spiral (user-centered design)• User research involved in waterfall method• Specify > Design > Build > Test• Specify: Competitive analysis, interviews, surveys• Design: Interviews to evaluate concepts and prototypes• Build: (no specific user research)• Test: Usability tests and site analytics.

What is the most important goal of a user research session

Establish rapport (Participants are safe, and you arethere to take care of them)

What is the main goal of conducting user research

Goal - To affect product requirements, features, specifications and design.• Reasons - To minimize risk for the company, and to satisfy user goals.• Quadrants of user research:Urgent Not UrgentImportant Important, Urgent (ex.,usability tests)Important, Not urgent (ex., userneeds assessments)Not important Not important, Urgent (CEOrequests)Not important, Not urgent (avoidthis area)

What is the NASA tasks load index

Multidimensional tool of six scales of perceived workload• Mental demand• Physical demand• Temporal demand• Frustration• Effort• Performance• Participants first rate the relative weight of the six scales via 15paired comparisons, and second rate the system on the six scalesusing 7-point scales.• The Index is computed by multiplying the relative weight of eachscale by the rating for each scale

What are two ways to visualize eye tracking data

Scan path - Sequence of locations where the eye gaze of one participant fixated Heat map - Amount of time multiple participants fixated on screen elements or areas

What is a usability test

Structured user interview focused on specific featuresin a user interface by focusing on task performance andsatisfaction

what are questions a user research must ask when choosing a user research method

What test should I use? What sample size do I need?• Do I need to do the computations by hand?

What is the first step in starting user research

make a plan starting with what the research will investigate During planning:• Gather research of internal discovery (Ask Sales, Tech Support, etc.)• Conduct surveys, usage data analysis, usability testing of currentproducts, observational field visits, focus groups, competitive analysis

What are four components of a human machine system

user, tool, task, environment

What is a semantic differential scale

7-point scale with bipolar anchors• (I usually use semantic differential scales)• Examples: Very difficult/Very easy, Weak/Strong, Ugly/Beautiful,Amateur/Professional

What is user experience

A situation that involves a human userinteracting with a system which is observable or measurable.• (If data is measuring only attitudes or preferences, then the data is notuser experience.)

How does a focus group moderator interact with observers

Before: Prepare the observers for what they are about to witness.• Before: Give a copy of the Moderator's Guide to each observer.• During: Ask observers if they have any questions for the group.• After: At conclusion of each group, lead a quick debriefing with theobservers on what they heard to get a group consensus ofresponses. Take notes from the debriefing.

What is the purouse of card soring in user research

Card sorting - Associative technique that invites participants togroup objects together, so that perceived similarities andrelationships can be made visible.• Types• Open - Participants can create their own title cards for their groupings• Closed - Participants must use pre-defined title cards for their groupings(e.g., pre-existing titles on a good website)• Ensure that the cards make sense to participants, can bereasonably grouped, have level of details, contain importantcontent, and are unbiased.• Number each card discreetly with a random number to facilitatedata compilation later.

Which tasks should be incldued in a usaiblty test

Choosing features to test - Tasks that are either: frequently used,new, troublesome, potentially dangerous, or important to users

What are questions a user research must ask when analyzing quantitative data

Comparing or correlating data?• Testing against a benchmark?• Task time?• Different users in each group?• Three or more groups to compare?

How does a user researcher analyze data from a usability test

Compiling - task times, ratings, errors, completion rates, quotes,comments• Organizing• Create a category for each response for each question.• Use your judgment if two responses are similar enough to be counted inthe same category. ("Like" and "enjoy", and "Dislike" and "hate")• Continue these judgments for each response and question.• Organizing is a method to reduce many responses into 5-10 categories• Extracting trends - Determine the user interface causes of eachcategory of responses.

How does a user researcher analyze task times

Convert all task time data to seconds and then performcalculations. Convert back to minutes and seconds for yourwritten report or presentation.• Number of decimals - Use one more significant digit incomparison to the original data. Range - Useful when checking for outliers and inaccurate data entry• Variance - Describes the variability• Standard deviation - Square root of variance, and in same units asoriginal data• The Excel Data Analysis ToolPak (Add-In) will display descriptivestatistics, but you must rerun the analysis every time you add or editdata.• A confidence interval decreases (a good thing!) as standarddeviation decreases and as sample size increases.

When should a user researcher conduct competitive research

Defining product requirements Redesigning your own product Reassessing a competitor's product redesign

What is the purpose of a diary study

Diary studies - Self-reported activities over time• I asked CAD users to keep notes on usage and what happened,and that we would discuss them over Zoom in 2 weeks' time.• Positive attributes:• Unobtrusive with no observers• Diary entry is written directly after event to reduce memory problems• Conduct all participants simultaneously• Capture both digital and real-world behavior

What regulations should a user researcher be aware of

General Data Protection Regulation (GDPR) for datacollection and privacy (Applicable whereverthere are citizens of the European Union)• Follow Institutional Review Boards (IRBs) - Usually just inuniversity and medical settings• Build your own usability lab or user research room

How does a user researcher recruit participants for user research

Hire a Market Research firm• Explore blogs of competitor companies. Note names of users ofcompetitive products and contact them Determine target user• Find them• Convince them to schedule (Determine most important incentive: money,pride, helping others, product improvement, etc. Frequency of a specific behavior (Times/day accessing web sites)• Amount of interest in a feature. (Do you want typical users or earlyadopters?)

What is an adequate number of participants in a usability test

If an iterative process and for an well-understood product, then n = 5 issuitable (Although I always schedule 8 participants to get 6)• If an expansive user interface with many tasks, then maybe n = 10 - 25

what is the proper scope of user research

Include ideas from marketing, sales, quality control, technicalsupport. Ever changing, and contains expectations, schedules,responsibilities, goals, outputs, budgets During planning:• Gather research of internal discovery (Ask Sales, Tech Support, etc.)• Conduct surveys, usage data analysis, usability testing of currentproducts, observational field visits, focus groups, competitive analysis• During development:• Conduct usability tests of design concepts and prototypes• During product sales:• Conduct post-release surveys, usage data analysis, diaries of usage andcomments, observational field visits. (Use this post-release data to feedinto the next Planning phase.)

How does a user researcher select particpants for user research

It is possible to compute valid statistics of sample sizes less than 10.• Be sure that your test participants are representative of the targetgroup, regardless of sample size

What is the purpouse of a field visit

Moen shower example did not mention the industrial designcompany Moen hired to conduct the research and to design theshowerhead. It was Design Continuum that was located inNewton, MA. I was their first HF manager and engineer.• People tend to idealize their needs and behaviors, so observingthem in their natural settings can uncover unspoken needs.• Needs: Patent (spoken, obvious), and Latent (unspoken, hidden)• Next slide is a civil engineer who uses CAD software in his office• What is on or near the user, and what is not?• What is displayed?• What information is printed out versus digital?• What is stuck on the walls and screens Field research reveals• Workarounds that users have developed. These workarounds revealdesign opportunities. Probe non-judgmentally how the workaroundstarted. (Look for Sticky Notes. What information is on them?)• Objects used for unintended purposes• Widespread rule breaking• Improvised physical interventions• Outcomes• Specifying concrete details about actual use• Uncovering hidden, forgotten goals and motivations• Challenging assumptions about the users

What are the characteristics of effective UX metrics

Observable• Quantifiable• Experiential: effectiveness, efficiency, and satisfaction• People-oriented• Suitable for any type of product or service• Be quick to collect• Be inexpensive• Help focus on small or large improvements• Help understand causes• Be cleaned up from extreme values• Replace gut feelings with actual data• Establish baselines for new products• Exist for most or all issues of concern• Be understood, tracked, and appreciated by management• Be reliable and usefully collected with small sample sizes UX metrics are used to make informed decisions. Easy to spotproblems, but need valid user research to estimate the populationproportion of users who will have that problem.• UX metrics stay mainly the same, though technologies advance.

what do the two administrators do during a user research session

One interacts with participant• One collects data in a format that facilitates data analysis (type directlyinto MS Excel)

What is the purpouse of using pictures and photos in user research

Photo elicitation - Dialogic technique that prompts conversationbetween researchers and participants• Source of images - User-generated (will take more time andincentives), or researcher-assembled.• Ask research questions in chronological or importance order.Don't show any pictures until after you introduce them. Showthem one at a time.• Instead of photos, you can ask participants to bring in objects orother artifacts (an anthropological term meaning object in timeand cultural meaning)

What is an effective task in a usaibilty test

Reasonable for target user• Described in terms of goals, not by step-by-step instructions• Specific so the participant can determine if task is complete• Doable and able to be completed• Follow a realistic sequence• Completable in 1-15 minutes (If there are subtasks embedded in task,then group the subtasks in a meaningful way) goal - rationale - action

How can a user researcher measure emotional reaction to a user interface

Self-reported comments - All seven emotions (could be all you need)• Verbal expressions - All seven emotions (could be all you need)• Eye tracking - Engagement• Galvanic skin response - Engagement and stress• Facial expressions - Engagement, joy and surprise Biometric data, galvanic skin response, psychoneuoimmunology

What is one form of a UX score card

Shows multiple metrics• Graphically present the results of the study in a summary chart.• Two metrics in a bar chart, and five metrics in a polar radar charttransformed into percentages or same scale (next slide)• However to preserve task data, use Harvey Balls of partially-filled circles

When should a user researcher conduct a focus group

Structured group discussion on preferences,experiences and priorities. Differentiator is that participants heareach other's responses.• Easy to invite stakeholders and development team to observeseveral groups in one day. Observers get lunch and candy!• Avoid having observers be remote, and observe just one focus group.• Advantages: fast to set up and analyze, probe responses• Disadvantages: out of context, misremembered or idealizedexperiences, non-representativeness of participants Exploratory - General attitudes and preferences• Feature Prioritization - Level of attractiveness of features• Competitive analysis - Value of competitors' products• Trend explanation - Investigation of motivations and expectations

What can be rated for severity in perfornmace metrics in a usabilty test

Task Success: 1 no problem, 2 minor problem, 3major problem, 4 failure Errors: Consider weighting the number of errors by severity (1 trivial, 2moderate, 3 serious)

What is a performance metiric

Task times, number of user actions, errors,proficiency time

What is usability

The ability of the user to use a system to complete auseful task successfully. More narrow than user experience.

What does a user researcher need to determine for a diary study

Timing of a diary study• Early in design to capture behaviors and feelings• Later in design as a form of extended remote usability test• Similar study would be an image diary study where you ask peopleto take pictures of events as they unfold. Provide them with adigital or single-use camera.• Prepare for a diary study• Create a customized digital or paper workbook for the study• Screen participants for those most likely to complete the days or weeks ofstudy• Begin research with more participants than you need because of drop-off

What is triangulation of user research

Use at least three metrics (e.g., performance,preferences, emotion) to gauge user and product success• A successful product should have a triangulation of fast task times,high value of system, and likely recommendation of product to others

What is formative user research

What works well? What works not so well? What arethe most common errors? What are the needed changes?• Should be conducted iteratively.