CSE 463 Final

Selection bias

selected subjects do not match population

Selection bias

subjects not selected randomly)

When To Use Speech Within UIs

- Mobile- Hands-Busy- Eyes-Busy- Assistive Technologies

Speech UIs consist of

- Speech Recognition Speech Production

Speech UI Design Guidelines

1. Show the system status 2. Match the concepts in the real world 3. Maximize user control and freedom 4. Be consistent and follow standards 5. Prevent errors 6. Expect users to recognize rather than recall 7. Enable flexibility and efficient operation 8. Apply minimal design 9. Help users recognize, diagnose, and recover from errors 10.Provide help and documentation

IKEA Effect

A increased sense of ownership and purpose (i.e. value) was experienced by the participants who successfully built the IKEA table. Friction was the effort required to build the table. They also discovered that value was destroyed if the box was too difficult to build (i.e. too much friction) .But too little friction also reduces the perception of value.

Metrics

A way of measuring/evaluating something or a particular phenomenon / aspect of something. A metric is a "system or standard of measurement", represented in units that can be utilized to describe more than one attribute. Usability is generally measured using observable and quantifiable metrics Metrics come in very handy when it comes to quantify usability during the usability evaluation of software, websites and applications.

Essential processing + incidental processing (caused by confusing presentation) > cognitive capacity

Aligning and Eliminating Redundancy needed: Aligning: Place printed words near corresponding parts of graphics to reduce need for visual scanning. Eliminating redundancy: Avoid presenting identical streams of printed and spoken words.

Von Restorff Effect

Also known as The Isolation Effect, predicts that when multiple similar objects are present, the one that differs from the rest is most likely to be remembered. Make important information or key actions visually distinctive.

Fitts Law

An empirical model of human motor performance that predicts that the time required to rapidly move to a target area is a function of the ratio between the distance to the target and the width of the target.

Temporal contiguity effect

Better information transfer occurs when corresponding animation and narration are presented simultaneously rather than successively.

Coherence effect:

Better information transfer occurs when extraneous material is excluded

Spatial contiguity effect:

Better information transfer occurs when printed words are placed near corresponding parts of graphics.

Signaling effect:

Better information transfer occurs when signals are included.

Modality effect:

Better information transfer occurs when words are presented as narration rather than as on-screen text.

Redundancy effect:

Better information transfer occurs when words are presented as narration rather than narration and on-screen text.

Technique for Collecting Data

Data always collected after hypotheses stated Data always collected after establishing decision criteria This sequence assures objectivity Compute a sample statistic (z-score) to show the exact position of the sample In words, z is the difference between the observed sample mean and the hypothesized population mean divided by the standard error of the mean.

External Validity

Degree to which findings can be inferred to the population of interest or to other populations or settings

Skeuomorphism

Design concept of making items represented resemble their real-world counterparts.

Testing Usability in the Field

Direct Observation Logging actual use Questionnaires and interviews with real users Focus Groups

Field Study Cons

Distractions Noise

Law of Common Region

Elements tend to be perceived into groups if they are sharing an area with a clearly defined boundary. Adding borders (creating common regions) around an element or group of elements is an easy way to create separation from surrounding elements.

Law of Uniform Connectedness

Elements that are visually connected are perceived as more related than elements with no connection. Group functions of a similar nature so they are visually connected via colors, lines, frames, or other shapes

Cognitive Overload - 3 Types of Demands

Essential Processing- Incidental Processing- Representational Holding

Behavioral and Physiological Metrics

Eye Tracking Measuring Emotion Stress and Other Physiological Measures

Rapid Prototyping

Goal: Produce a higher precision prototypes than can be achieved with offline techniques. Aims to collect information on requirements and the adequacy of possible designs. Recognizes that requirements are likely to be inaccurate when first specified Emphasis is on evaluating the design before discarding i

Law of Proximity

Objects that are near, or proximate to each other, tend to be grouped together. The law of proximity is useful by allowing users to group different clusters of content at a glance.

Spatial ability effect:

High spatial learners benefit more from well-designed instructions than do low spatial learners.

Threats to Internal Validity

History Maturation Testing Instrumentation Statistical regression Selection Bias Experimental Mortality Interaction among factors

System Model vs Interaction Model

How something works vs. How to use something

Value of Inconvenient Design

Humans are friction obsessed. Friction is our ultimate foe in a constant crusade for efficiency and optimization. It slows us down and robs us of energy and momentum. It makes things hard. We dream of futures where things run smoothly and effortlessly, where it is all so easy.

Law of Pragnanz

People will perceive and interpret ambiguous or complex images as the simplest form possible, because it is the interpretation that requires the least cognitive effort of us. The human eye likes to find simplicity and order in complex shapes because it prevents us from becoming overwhelmed with information.

Laboratory Experiment Pros

Specialist Equipment available Uninterrupted environment

Essential processing + representational holding > cognitive capacity.

Synchronizing and Individualizing needed: Synchronizing: Present narration and corresponding animation simultaneously to minimize need to hold multiple representations in memory. Individualizing: Make sure learners possess skills for holding mental representations.

Performance Metrics

Task Success (Completion Rates)• Binary Success • Levels of Successo Time on Tasko Errors Effectiveness Efficiency Learnability

Purpose of Laboratory Experiment

Test Hypotheses derived from theory Study the precise interrelations of variables and their operation Control variance

Millers Law

The average person can only keep 7 (plus or minus 2) items in their working memory. Chunking is an effective method of presenting groups of content in a manageable way. Organize content in groups of 5-9 items at a time.

Instrumentation

changes in raters, different focus / agreement between observers

Prototype

concrete representation (partial or complete) of an interactive system. a tangible artifact, not an abstract description that requires interpretation. goal of prototyping is to test and inspect usability, and to detect and correct potential design failures as early as possible. designers, managers, developers, customers, and end users can use these artifacts to envision and reflect on the final system.

High Fidelity Prototyping

for testing details specific functionality more functionality, closer to the final product easier to test, as users can interact with the prototype expensive and time consuming reluctance to change (due to high costs)

Experimental mortality

participant change over time

Interaction Effects

pre-testing may tip off participants

Reactive effect of experimental setting

treatment in laboratory may not be effective in real world

Design Guidelines: Search Functionality

• Provide a search option on each page

Hypothesis Testing

Hypothesis testing a statistical method that uses sample data to evaluate the validity of a hypothesis about a population parameter.

Uncertainty and Errors in Hypothesis Testing

Hypothesis testing is an inferential process. Uses limited information from a sample to make a statistical decision, and then from it a general conclusion. Sample data used to make the statistical decision allows us to make an inference and draw a conclusion about a population. Errors are possible.

ICAP

ICAP = Interactive > Constructive > Active > Passive Rather than focusing solely on the outcomes of activities, the teacher seeks to measure whether students actively engage in the learning process along the way.

Technique for Making a Decision

If sample statistic (z) is located in the critical region, the null hypothesis is rejected. If the sample statistic (z) is not located in the critical region, the researcher fails to reject the null hypothesis.

Essential processing in visual channel > cognitive capacity of visual channel.

Off-loading needed: Move some essential processing from visual channel to auditory channel.

Ecological Validity

Results of an experiment can be generalized from the set of environmental conditions in the experiment to other environmental conditions

Self-Reported Metrics

Information about users' perception of the system Two Types of rating scales (likert and semantic) Types of self-report metrics (post task, post session, usability scale, online services)

Laboratory Experiment Cons

Lack of context Difficult to observe several users cooperating

Hick's Law

Models human reaction time under uncertainty. States that decision time increases with uncertainty about the judgment or decision to be made.

Parallel Prototyping

Multiple design concepts are evaluated before beginning an iterative design process

Iterative Prototypes

Multiple iterations of a three-step process:- Prototype- Review- Refine

Field Study Pros

Natural environment Context retained

Technique For Stating Hypothesis

Null Hypothesis (Ho) means no change Alternative Hypothesis (H1) means there is a change

Four Rules of Intuitive UX

Obey the Law of Locality ABD: Anything But Dropdowns Pass the Squint Test Teach by example

Field Experiment

Research study conducted in a realistic situation in which one or more independent variables are manipulated No major distinctions between laboratory and field

Type 2 Error

Researcher fails to reject a null hypothesis that is really false. Researcher has failed to detect a real treatment effect.

Type 1 Error

Researcher rejects a null hypothesis that is actually true. Researcher concludes that a treatment has an effect when it has none.

Rapid Prototyping Techniques

Paper and Pencil Sketches Mockups Wizard of Oz - virtual assistant Video Prototyping - can be used to tell a story

Peak-End Rule

People judge an experience largely based on how they felt at its peak and at its end, rather than the total sum or average of every moment of the experience.

Zeigarnik Effect

People remember uncompleted or interrupted tasks better than completed tasks. Use progress bars for complex tasks to visually indicate when a task is incomplete, and thus increase the likelihood it will be completed.

Mental Model

Perception or Representation that a person has in his mind of the product interacting with People have sets of experiences built up, and they take those experiences into this new product you are making. That is how/why they form expectations on the product you are building. Mental Models can evolve over time. The gap between developers and the users needs to be brought closer. Developers need to understand a culture's various mental models and core values to truly build a user-friendly experience.

Metric Types

Performance Metrics Issue Based Metrics Self Reported Metrics Behavioral and Physiological Metrics Combined and Comparative Metrics

Hofstede's Cultural Dimensions

Power distance Uncertainty avoidance Individualism vs. collectivism Masculinity vs. femininity Long vs. short-term time orientation Indulgence vs. restraint

Horizontal Prototypes

Purpose: to develop an entire layer of the design at the same time. Provide a wide range of functions, but with little detail.

Vertical Prototypes.

Purpose: to ensure that the designer can implement the full, working system from the user interface layer down to the underlying system layer. Provides in-depth functionality for a few selected features. - Task-Oriented Prototypes. - Scenario-Based Prototypes.

Jakob's Law

Recognizes that users spend most of their time on other sites This means that users prefer your site to work the same way as all the other sites they already know. You can simplify the learning process for users by providing familiar design patterns.

Population Validity

Refers to the extent to which the results can be generalized from the experimental sample to a defined population

Technique for Decision Criteria

Sample outcomes divided for likely Ho true or unlikely H1 true Alpha level is significance level used to define very unlikely outcomes critical regions consist of the extreme sample outcomes that are unlikely

Essential processing in both visual and auditory channels > cognitive capacity.

Segmentation and Pretraining needed: Segmentation effect: Better information transfer occurs when lesson is presented in learner-controlled segments rather than as continuous unit. Pretraining effect: Better information transfer occurs when students know names and behaviors of system components.

Storyboards

Series of sketches showing how a user might progress through a task using the device

Combined and Comparative Metrics

Sometimes it is easier to describe the usability of a system or task by combining metrics into a single score (e.g. comparing competing products or reporting on corporate dashboards). SUM is a standardized average of measures of effectiveness, efficiency of satisfaction and is typically composed of 3 metrics: completion rates, task-level satisfaction and task time. Single Usability Scores Comparison to Goals and Expert Performance

Speech UI Operational Design Characteristics To Avoid

Speech modes where the user receives no feedback and where certain voice commands only work in certain states.- Deep speech hierarchy trees the user must navigate

Difficulties With Speech UIs

Speech recognition technology is far from perfect- Speech UIs have no visible user status displays- Speech UIs are hard for users to learn- Isolated and short words pose difficulty - Segmentation- Spelling for identically pronounced words- Context of user interaction is often difficult to determine

Process Steps

Step 1: State the hypotheses. Step 2: Set the criteria for a decision (predict the expected characteristics of the sample based on the hypothesis) Step 3: Collect data (obtain a random sample from the population); compute sample statistics. Step 4: Make a decision. Compare the obtained sample data with the prediction made from the hypothesis.• If consistent results obtained, hypothesis is reasonable • If discrepant results obtained, hypothesis is rejected

Friction Value Curve

The challenge for developers is to build systems where users straddle the friction-value curve. On one hand, the system cannot be too challenging to cognitively overload or make the action of completing a task so difficult that the user simply gives up. However, if the system is too easy (i.e. too little friction) it enters the realm of being perceived as without purpose or meaning, leaving the user unsatisfied.

Law of Similarity

The human eye tends to perceive similar elements in a design as a complete picture, shape, or group, even if those elements are separated. Ensure that links and navigation systems are visually differentiated from normal text elements and are consistently styled.

Quality of Designs

The parallel design process yielded better-performing designs

Design Diversity

The parallel process generated more diverse designs

Directional Hypothesis Tests

The standard hypothesis testing procedure is called a two-tailed (non-directional) test because the critical region involves both tails to determine if the treatment increases or decreases the target behavior. However, sometimes the researcher has a specific prediction about the direction of the treatment. When a specific direction of the treatment effect can be predicted, it can be incorporated into the hypotheses. In a directional (one-tailed) hypothesis test, the researcher specifies either an observed increase or a decrease in the population mean resulting from the treatment.

Bloom's Taxonomy

This taxonomy presents an easily digestible framework for understanding how deeper levels of thinking build on foundations of more simplified forms of thinking.

Issue-Based Metrics

Usability problems Purely qualitative Usability issues are based on behavior in using a product.• Behaviors that prevent task completion. Behaviors that takes someone "Off course". An expression of frustration by the participant. Not seeing something that should be noticed. A participant says a task is complete when it is not Performing an action that leads away from task success. Misinterpreting some piece of content. Choosing the wrong link to navigate through web pages.

Design Guidelines: Usability Testing

Use an iterative design approach

Wireframes

Used early in the development process to establish the basic structure of a page before visual design and content is added

Doherty Threshold

User productivity soars when a computer and its users interact at a pace (<400ms) that ensures that neither has to wait on the other.

Serial Position Effect

Users have a propensity to best remember the first and last items in a series. Placing the least important items in the middle of lists can be helpful because these items tend to be stored less frequently in long-term and working memory. Positioning key actions on the far left and right within elements such as navigation can increase memorization.

Aesthetic Usability Effect

Users often perceive aesthetically pleasing design as design that's more usable.

Internal Validity

Validity of findings with the research study

Essential processing + incidental processing (caused by extraneous material) > cognitive capacity

Weeding and Signaling needed: Weeding: Eliminate interesting, but extraneous material to reduce processing of extraneous material. Signaling: Provide cues for how to process the material to reduce processing of extraneous material.

Self-Efficacy

alternative designs encourage investment in a creative process rather than one particular idea

History

distraction occurring during the experiment

Maturation

fatigue, hunger

Low Fidelity Prototyping

for testing ideas and sequences fast, cheap, easy to change, throw-away very communicative as all parties involved can be included (everybody can draw, no need to be a professional graphic designer) No/little functionality limited ability to detect errors Not all ideas might be technically feasible

Statistical regression

re-testing of extreme individuals often improve results

Mock-up

rudimentary throw-away prototype software that will look like the UI without having to build the software or the underlying functionality.

Multiple treatment interference

users receiving more than one treatment may influence subsequent treatments

Interaction among factors

various experimental factors can influence results