User-Experience Research Methods

UX Methods in Brief: Diary/Camera Studies

Participants are given a mechanism (diary or camera) to record and describe aspects of their lives that are relevant to a product or service, or simply core to the target audience; diary studies are typically longitudinal and can only be done for data that is easily recorded by participants.

UX Methods in Brief: Participatory Design

Participants are given design elements or creative materials in order to construct their ideal experience in a concrete way that expresses what matters to them most and why.

UX Methods in Brief: Desirability Studies

Participants are offered different visual-design alternatives and are expected to associate each alternative with a set of attributes selected from a closed list; these studies can be both qualitative and quantitative.

The Qualitative vs. Quantitative Research Dimension

- Qualitative methods are better suited for answering questions about why or how to fix a problem. - Quantitative methods so a much better answering how many and how much types of questions.

UX Methods in Brief: A/B Testing (related to "multivariate testings," "live testing," or "bucket testing")

A method of scientifically testing different design on a site by randomly assigning groups of users to interact with each of the different designs and measuring the effect of these assignments on user behavior.

UX Methods in Brief: True-Intent Studies

A method that asks random site visitors what their goal or intention is upon entering the site, measures their subsequent behavior, and asks whether they were successful in achieving their goal upon exiting the site.

UX Methods in Brief: Unmoderated Remote Panel Studies

A panel of trained participants who have video recording and data collection software installed on their own personal devices uses a website or product while thinking aloud, having their experience recorded for immediate playback and analysis by the researcher or company.

UX Methods in Brief: Email Surveys

A survey in which participants are recruited from an email message.

UX Methods in Brief: Intercept Surveys

A survey that is triggered during the use of a site or application.

Three-Dimensional Framework

Attitudinal vs. Behavioral, Qualitative vs. Quantitative, and Content of Use.

UX Methods in Brief: Customer Feedback

Open-ended and/or close-ended information provided by a self-selected sample of users, often through a feedback link, button, form or email.

UX Methods in Brief: Usability-Lab Studies

Participants are bought into a lab, one-on-one with a researcher, and given a set of scenarios that lead to tasks and usage of specific interest within a product or service.

Qualitative Research

Studies that are qualitative in nature generate data about behaviors or attitudes based on observing the directly. - example: the researcher directly observes how people use technology (or not) to meet their needs. This gives them the ability to ask questions, probe on behavior, or possibly even adjust the study protocol to better meet its objective. Analysis of the data is usually not mathematical.

UX Methods in Brief: Usability Benchmarking

Tightly scripted usability studies are performed with several participants, using precise and predetermined measures of performance.

Behavioral Research

"What People Do" The purpose of behavioral research focuses mostly on seeking to understand "what people do" with product or service in question. - methods include: A/B testings (ex. changing of website for selected group and not others), Eyetracking (understand how users visually interact with interface designs).

Attitudinal Research

"What People Say" The purpose of attitudinal research is used heavily to understand or measure people's stated beliefs, which is why attitudinal research is used heavily in marketing departments. - methods include: self-reported information, card sorting (provides insights about users' mental model of an information space/can help determine the best information architecture for product/application/website), and surveys (measure and categorize attitudes or collect self-report data that can track or discover important issues to address). - Focus groups tend to be less useful, but provide a top-of-mind of what people think about a brand or product concept in a group setting.

UX Methods in Brief: Eyetracking

An eyetracking device is configured to precisely measure where participants look as they perform tasks or interact naturally with websites, applications, physical products, or environments.

UX Methods in Brief: Clickstream Analysis

Analyzing the record of screens or pages that users clicks on and sees, as they use a site or software product; it requires the site to be instrumented properly or the application to have telemetry data collection enabled.

UX Methods in Brief: Unmoderated UX Studies

A quantitative or qualitative and automated methods that uses a specialized research tool to captures participant behaviors (through software installed on participant computer/browsers) and attitudes (through embedded survey questions). usually by giving participants goals or scenarios to accomplish with a site or prototype.

UX Methods in Brief: Card Sorting

A quantitative or qualitative method that asks users to organize items into groups and assign categories to each group. This method helps create or refine the information architecture of a site by exposing users' mental models.

UX Methods in Brief: Interviews

A researcher meets with participants one-to-one to discuss in depth what the participant thinks about the topic in question.

UX Methods in Brief: Concept Testing

A researcher shares an approximation of a product or service that captures the key essence (the value proposition) of a new concept or product in order to determine if it meets the needs of the target audience; it can be sone one-on-one or with larger numbers of participants, and either in person or online.

The Context of Product Use: Scripted Use

Focuses on the insights of specific usage aspects, such as on a newly designed flow. - degree of scripting can vary quite a bit, depending on the study goals. (example: a benchmarking study is usually very tightly scripted and more quantitative in nature, so that it can produce reliable usability metrics.)

UX Methods in Brief: Focus Groups

Groups of 3-12 participants are lead through a discussion about a set of topics, giving verbal and written feedback through discussion and exercises.

The Context of Product Use: Natural Use

Minimizing the interference from the study in order to understand behavior or attitudes as close to reality as possible. - provide greater validity but less control over what topics you learn about, Intercept survey and data mining or other analytic techniques are quantitative examples of this.

UX Methods in Brief: Ethnographic Field Studies

Researchers meet with and study participants in their natural environment, where they would most likely encounter the product or service in question.

Quantitative Research

Studies that are quantitative in nature generate data about behavior or attitudes in questions are gathered indirectly (ex. quantitative studies), through a measurement or an instrument such as a survey or an analytics tool. - quantitative methods are typically derived from mathematical analysis, since the instrument of data collection (e.g. survey tool or web-server log) captures such large amounts of data that are easily coded numerically.

The Context of Product Use

The third distinction has to do with how and whether participants in the study are using the product or service in question: - Natural or near-natural use of the product - Scripted use of the product - Not using the product during the study - A Hybrid of the above

The Attitudinal vs. Behavioral Research Dimension

This distinction can be summed up by contrasting "what people say" versus "what people do." - Between these two extremes lie the most popular methods used: usability studies and field studies. They utilize a mixture of self-reported and behavioral data, and can move toward the behavioral side is generally recommended.

UX Methods in Brief: Moderated Remote Usability Studies

Usability studies conducted remotely with the use of tools such as screen-sharing software and remote control capabilities.

20 UX Methods in Brief

Usability-Lab Studies, Ethnographic Field Studies, Participatory Design, Focus Groups, Interviews, Eyetracking, Usability Benchmarking, Moderated Remote Usability Studies, Unmoderated Remote Panel Studies, Concept Testing, Diary/Camera Studies, Customer Feedback, Desirability Studies, Card Sorting, Clickstream Analysis, A/B Testing, Unmoderated UX Studies, True-Intent Studies, Intercept Surveys and Email Surveys.

The Context of Product Use: Not Using

Where the product is not used or conducted to examine issues that are broader than usage and usability, such as a study of the brand or larger cultural behaviors.