463 Part II of Semester Material

an example of Interactive engagement would be....

Examples of these activities may include debates, justifying a position in pairs or small groups, or having students gage each other's understanding through questioning (Chi & Wylie, 2014).

T/F if the sample statistic is located in the critical region the null hypothesis is accepted

False. If sample statistic (z) is located in the critical region, the null hypothesis is rejected.

Online rapid prototyping techniques

Goal: create higher precision prototypes that can be achieved with offline techniques o non-interactive simulations/ parallel prototyping o interactive simulations

ICAP stands for

Interactive > Constructive > Active > Passive

In the ICAP style, the actions or thinking can be categorized into four levels:

Interactive > Constructive > Active > Passive

hypothesis testing

a statistical method that uses sample data to evaluate the validity of a hypothesis about a population parameter.

Boundaries of critical region(s) are determined by the probability set by the _________ level.

alpha

null hypothesis

a statement or idea that can be falsified, or proved wrong

SUS summary

*0-100 Scores are not percentages *Measures usability and learnability *Reliable and valid *NOT diagnostic *Modest correlation with task performance

Scoring SUS

*For odd items: subtract one from the user response. *For even-numbered items: subtract the user responses from 5 *This scales all values from 0 to 4 (with four being the most positive response). *Two summary score options *Add up the converted responses for each user and multiply that total by 2.5. This converts the range of possible values from 0 to 100 instead of from 0 to 40. *Add up the converted responses for each user and divide by 10 to get average response

SUS 1-5

*The SUS is a 10 item questionnaire with 5 response options. 1.I think that I would like to use this system frequently. 2.I found the system unnecessarily complex. 3.I thought the system was easy to use. 4.I think that I would need the support of a technical person to be able to use this system. 5.I found the various functions in this system were well integrated.

sus

*The System Usability Scale (SUS) was released into this world by John Brooke in 1986. *It was originally created as a "quick and dirty" scale for administering after usability tests on systems like VT100 Terminal ("Green-Screen") applications. *SUS is technology independent and has since been tested on hardware, consumer software, websites, cell-phones, IVRs and even the yellow-pages. *It has become an industry standard with references in over 600 publications.

SUS 6-10

6.I thought there was too much inconsistency in this system. 7.I would imagine that most people would learn to use this system very quickly. 8.I found the system very cumbersome to use. 9.I felt very confident using the system. 10.I needed to learn a lot of things before I could get going with this system.

mental model

A Mental Model is the perception or representation that a person has in his mind of the product he is interacting with.

Advantages and Disadvantages of mock-ups

Advantages •Organize project details - Visual hierarchies, color, font, etc. •Finds errors early on •Translates ideas into a stakeholder language •Communicates ideas between team members •Design implementation •The user's perspective Disadvantages •Requires substantial time and resources.

_______ is a probability value used to define "very unlikely" outcomes.

Alpha level or significance level

14 metrics

Performance Metrics 1.Completion rate (task success, effectiveness) 2.Task time 3.Errors 4.Efficiency (Page views/clicks) 5.Lostness 6.Conversion rate 7.Learnability 8.Eye-tracking 9.Biometric data Combined Metrics 10.Single usability metric* Self-Report Metrics 11.Task-level satisfaction (self-report) 12.Expectations 13.Test-level satisfaction (self-report) Issue-Based Metrics 14.Usability problems

Type 1 Error

Researcher rejects a null hypothesis that is actually true. -> Researcher concludes that a treatment has an effect when it has none.

hypothesis steps

Step 1: State the hypotheses. Step 2: Set the criteria for a decision. Step 3: Collect data; compute sample statistics. Step 4: Make a decision.

an example of constructive engagement would be....

Students at this stage may take notes in their own wording, create metaphors, compare with outside sources, and integrate previous thinking with current conversation

SUS

System usability scale

alternative hypothesis

The hypothesis that states there is a difference between two or more sets of data. Contradicts null.

T/F Data is always collected after hypotheses stated and establishing decision criteria are established

True

T/F ICAP is a framework seeking to depict the process of engaged learning by Chi and Wylie's (2014) ICAP Hypothesis.

True

T/F errors are possible in hypothesis testing

True

T/F Hypothesis testing is an inferential process.

True o Uses limited information from a sample to make a statistical decision, and then from it a general conclusion. o Sample data used to make the statistical decision allows us to make an inference and draw a conclusion about a population.

•Rather than focusing solely on the outcomes of activities, the teacher seeks to measure whether students actively ___________ in the learning process along the way.

engage

type 2 error

failing to reject a false null hypothesis -> Researcher has failed to detect a real treatment effect.

Number and types of metrics

o 5-14 metrics o types: performance, issue-based, self-reported o metrics vary in complexity with some that are simply binary variables and require researchers to calculate a sum to lostness that involves a more complicated formula

Von Restorff effect

o Also known as The Isolation Effect, predicts that when multiple similar objects are present, the one that differs from the rest is most likely to be remembered. o Make important information or key actions visually distinctive.

Step 3: Technique for Collecting Data

o Data always collected after hypotheses stated o Data always collected after establishing decision criteria o This sequence assures objectivity o Compute a sample statistic (z-score) to show the exact position of the sample o In words, z is the difference between the observed sample mean and the hypothesized population mean divided by the standard error of the mean.

Humans are friction-obsessed.

o Friction is our ultimate foe in a constant crusade for efficiency and optimization. o It slows us down and robs us of energy and momentum. o It makes things hard. We dream of futures where things run smoothly and effortlessly, where it's all so easy. o The challenge is to build the system to straddle the friction-value curve o On one side, we have the market-driven techno-industrial complex, which is focused on making things increasingly easier for people who are already in the sweet spot of the curve. o The result is that these people are beginning to slip down the other side, falling into the realm of too little friction and therefore leaving purpose, meaning, and satisfaction behind.

Step 4: Technique for Making a Decision

o If sample statistic (z) is located in the critical region, the null hypothesis is rejected. o If the sample statistic (z) is not located in the critical region, the researcher fails to reject the null hypothesis.

self-reported metrics

o Information about users' perception of the system o Two Types of rating scales • Likert Scales (5-point scale - strongly agree, agree, neutral, disagree, strongly disagree) • Semantic Differential Scales o Types of self-report metrics • Post-Task Ratings • Post session Ratings • Using System Usability Scale (SUS) to compare designs (10 item questionnaire) • Online Services

4 rules of intuitive ux design

o Obey the Law of Locality o ABD: Anything But Dropdowns o Pass the Squint Test o Teach by example

People have sets of experiences

o People have sets of experiences built up, and they take those experience into this new thing that you're making. And they expect it to work based on their previous expectations.

Peak-end rule

o People judge an experience largely based on how they felt at its peak and at its end, rather than the total sum or average of every moment of the experience. o Pay close attention to the most intense points and the final moments (the "end") of the user journey.

Hofstede's Cultural Dimensions

o Power distance o Uncertainty avoidance o Individualism vs. collectivism o Masculinity vs. femininity o Long vs. short-term time orientation o Indulgence vs. restraint o defined culture as set go shared characteristics, such as thoughts, values and behaviors

performance metrics

o Task Success (Completion Rates) • Binary Success • Levels of Success o Time on Task o Errors o Effectiveness o Efficiency o Lostness (analyzes efficiency data) o Learnability o Eye-tracking

issue-based metrics

o Usability problems o Purely qualitative o Usability issues are based on behavior in using a product. • Behaviors that prevent task completion. • Behaviors that takes someone "Off course". • An expression of frustration by the participant. • Not seeing something that should be noticed. • A participant says a task is complete when it is not • Performing an action that leads away from task success. • Misinterpreting some piece of content. • Choosing the wrong link to navigate through web pages.

doherty threshold

o User productivity soars when a computer and its users interact at a pace (<400ms) that ensures that neither has to wait on the other.

serial position effect

o Users have a propensity to best remember the first and last items in a series. o Placing the least important items in the middle of lists can be helpful because these items tend to be stored less frequently in long-term and working memory. o Positioning key actions on the far left and right within elements such as navigation can increase memorization.

Aesthetic Usability Effect

o Users often perceive aesthetically pleasing design as design that's more usable.

Measuring emotion

o affective and the Q-sensor o FACET o EEG o AFFDEX

Rapid Prototyping

o aims to collect information on requirements and the adequacy o recognizes that requirements are likely to be inaccurate when first specified o emphasis is on evaluating the design before discarding it

measuring emotion

o combination of probing questions, interpretation of facial expressions, and even body language to infer the participant's emotional state

prototype

o concrete representation (partial or complete) of an interactive system; early state of design. o a tangible artifact, not an abstract description that requires interpretation. o goal of prototyping is to test and inspect usability, and to detect and correct potential design failures as early as possible. o designers, managers, developers, customers, and end users can use these artifacts to envision and reflect on the final system.

advantages of wireframing

o early visual to review with client and can give early feedback for prototype usability tests o easier to amend than concept designs o ensure page content and functionality are positioned correctly based on user and business needs. as project moves forward they can be used as a good dialogue between members of the project team to agree with project vision and scope.

High Fidelity Prototype

o for testing details specific functionality o more functionality, closer to the final product o easier to test, as users can interact with the prototype o expensive and time consuming è reluctance to change (due to high costs) Ex. Axure and iRise

Low Fidelity Prototype

o for testing ideas and sequences o fast, cheap, easy to change, throw-away o very communicative as all parties involved can be included (everybody can draw, no need to be a professional graphic designer) o No/little functionality è limited ability to detect errors o Not all ideas might be technically feasible Ex. paper prototypes

Prototyping Strategies

o horizontal o vertical o task-oriented o scenario-based

Offline rapid prototyping techniques

o paper and pencil sketches o mock-ups o wizard of oz o video prototyping

Stress and other Physiological Measures

o participants might feel stressed when they have difficulty finding important information or when they are unsure about transaction they are going through o Heart Rate Variance o Heart Rate Variance and Skin Conductance Research (GSR - Galvanic Skin Response) o Other Measures (e.g. Pressure Mouse)

4 dimensions to analyze prototypes

o representation o precision o interactivity o evolution

mock-ups

o rudimentary throw-away prototype software that will look like the UI without having to build the software or the underlying functionality. o mock-up is a prototype if it provides part of the functionality of a system and enables testing of a design. o visual way of representing a product o shows what product will look like, mockup is not clickable o either mid or high-fidelity display or design

Storyboards should convey...

o setting: people involved, environment, task being accomplished o sequence o satisfaction

disadvantages of wireframing

o since it does not include any design or account for technical implication, not always easy for client to grasp the concept. designer will have to translate wireframes into design so communication to support wireframe is often needed to explain why page elements are positioned as they are o when content is added it might initially be too much to fit within the wireframe layout, so designer and copywriter will need to work closely to make this fit.

paper and pencil sketches

o sketching is important to low-fidelity prototyping o don't be inhibited about drawing ability sketches and screen mock-ups; quick to build and easy to run o users love paper prototypes; gives opportunity to contribute to new design

Prototypes as design artifacts

o support creativity o encourage communication o permit early evaluation

storyboards

o used with scenarios, bringing more detail, chance to role play o sequence of screens focusing on user action don't capture every detail, just main functionality o could be limited in scope, more rigidly linear o series of sketches showing how user might progress through a task using device o used in early design prototyping

Wireframes

o way to design website service at structural level wireframe commonly used to lay out content and functionality on page that takes into account user's needs and user's journeys o used in early development process establish the basic structure of a page before visual design and content is added o essential to UI design o much easier to adapt than a concept design

an example of Active engagement would be....

verbatim notes, highlighting a text, or perhaps rewinding or pausing a video

Critical region(s) consist of the extreme sample outcomes that are ______________

very unlikely

an example of passive engagement would be....

watching a video watching a lecture

_________ is the difference between the observed sample mean and the hypothesized population mean divided by the standard error of the mean.

z

Active Level:

• Focuses on active engagement. • Chi and Wylie characterize an activity as "active" if students participate in the direct manipulation of instructional materials or activities. No original thoughts however.

System model vs. interaction model

• System Model: How something Works • Interaction Model: How to use something o Good System Model o Bad Interaction Model

Parallel Prototyping

•"Parallel Prototyping" is where multiple design concepts are evaluated before beginning an iterative design process. • •There are two different flavors of parallel prototyping (or parallel design): •Several designers each create one or more designs, then the entire group considers all concepts. The best ideas are cherry picked to create one or more improved designs. •One designer creates multiple designs, then all designs are critiqued and the designer creates revised designs.

Fitts's Law

•An empirical model of human motor performance explaining speed-accuracy tradeoff characteristics of human muscle movement. •Aimed at arm-hand movement •Original model developed in 1954 •Predicts that the time required to rapidly move to a target area is a function of the ratio between the distance to the target and the width of the target. •Enables prediction of movement time (MT). •Movement assumed to be rapid, error-free, and targeted. •MT is a function of target distance and width. •Important in pointing tasks such as pointing speed, target distance, target size and accuracy. •Provides us a way to compare speed across tasks, limbs and devices.

0 Bloom's Taxonomy

•Bloom's Taxonomy maintains a central role in conversations about promoting higher order thinking skills in students. •This taxonomy presents an easily digestible framework for understanding how deeper levels of thinking build on foundations of more simplified forms of thinking. Knowledge, understanding, application, analysis, synthesis, evaluation

disadvantages of rapid prototyping

•Can consume a lot of resources - users, analysts, and programmers. Therefore, Rapid Prototyping can be costly as well as time consuming. •The continued process of design evaluate redesign may mean that the design phase of the cycle is considerably increased. •May be a long time before get a working system. •Reluctance to 'throw away' or discard the prototype. •Users expectations/wishes may be unrealistic. •May not be technically or economically feasible

Miller's Law

•The average person can only keep 7 (plus or minus 2) items in their working memory. •Chunking is an effective method of presenting groups of content in a manageable way. •Organize content in groups of 5-9 items at a time.

Interactive Level:

•Centers on interactive engagement. •At this final level, students build on their previous "constructive" thinking and emerge as a community of learners. The classroom sees students taking turns as active contributors to the broader understanding, and original thinking builds as multiple members contribute over time.

Caveat

•Despite the wide usage of SUS, there has been little guidance on interpreting SUS scores, acceptable modifications to the items and information on reliability and validity. •What is a Good SUS Score? •The average SUS score from 500 studies is a 68. •A SUS score above a 68 would be considered above average and anything below 68 is below average.

Step 2: Set the Decision Criterion

•Distribution of sample outcomes is divided: •Those likely if Ho is true •Those "very unlikely" if Ho is true •Alpha level, or significance level, is a probability value used to define "very unlikely" outcomes. •Critical region(s) consist of the extreme sample outcomes that are "very unlikely". •Boundaries of critical region(s) are determined by the probability set by the alpha level

Law of common region

•Elements tend to be perceived into groups if they are sharing an area with a clearly defined boundary. •Adding borders (creating common regions) around an element or group of elements is an easy way to create separation from surrounding elements.

Law of uniform connectedness

•Elements that are visually connected are perceived as more related than elements with no connection. •Group functions of a similar nature so they are visually connected via colors, lines, frames, or other shapes.

Constructive Level:

•Focuses on constructive engagement. •Rather than regurgitating instructional materials, students create novel ideas informed by personal experience, prior knowledge, and connections to broader areas of study.

Passive Level:

•Focuses on passive engagement. •On the passive level, activities promote a transactional notion of education, where students receive information with no expectation of interaction. At this stage, envision students watching a video or receiving a lecture.

Applied to UX design

•Gestalt principles of perceptual organization. •Law of Proximity •Law of Similarity •Law of Common Region •Law of Continuity •Law of Closure •Law of Focal Point

Advantages of rapid prototyping

•Helps the designer to evaluate the design very early in the design cycle. •It is good for addressing the problem of users not knowing or being unable to state their requirements. •Provides the opportunity for continued evaluation and refinement of the design. •Increases the chance of getting a well-designed system acceptable to the users with the required functionality and ease of use.

Hick's Law

•Hick's Law (or the Hick-Hyman Law) is named after a British and an American psychologist team of William Edmund Hick and Ray Hyman. •In 1952, this pair set out to examine the relationship between the number of stimuli present and an individual's reaction time to any given stimulus. •Models human reaction time under uncertainty. •Decision time increases with uncertainty about the judgment or decision to be made.

Hick's Law cont'd

•Hick's Law is a simple idea that says that the more choices you present your users with, the longer it will take them to reach a decision. •As a designer, you will use Hick's Law to examine how many functions you should offer at any part of your website and how this will affect your users' overall approach to decision making.

Fitt's Law Cont'd

•It states that the amount of time required for a person to move a pointer (e.g., mouse cursor) to a target area is a function of the distance to the target divided by the size of the target (i.e., the longer the distance and the smaller the target's size, the longer it takes). •Fitts' law is widely used in user experience (UX) and user interface (UI) design. •For example, this law influenced the convention of making interactive buttons large (especially on finger-operated mobile devices)—smaller buttons are more difficult (and time-consuming) to click. •Likewise, the distance between a user's task/attention area and the task-related button should be kept as short as possible.

Step 1: State Hypotheses - what are the two types?

•Null hypothesis (Ho) states that, in the general population, there is no change, no difference, or is no relationship. •Alternative hypothesis (H1) states that there is a change, a difference, or there is a relationship in the general population.

Survey Instruments cont'd 2

•Online surveys avoid the cost of printing and the extra effort needed for distribution and collection of paper forms •Many people prefer to answer a brief survey displayed on a screen, instead of filling in and returning a printed form •although there is a potential bias in the sample •A survey example is the Questionnaire for User Interaction Satisfaction (QUIS) •http://lap.umd.edu/quis/ •There are others, e.g. Mobile Phone Usability Questionnaire (MPUQ)

Mental model example

•People in the Middle East write from right to left, whereas people in the West write from left to right. •Developers need to understand this in order to build a usable website.

Zeigarnik effect

•People remember uncompleted or interrupted tasks better than completed tasks. •Use progress bars for complex tasks to visually indicate when a task is incomplete, and thus increase the likelihood it will be completed.

Law of Prägnanz

•People will perceive and interpret ambiguous or complex images as the simplest form possible, because it is the interpretation that requires the least cognitive effort of us. •The human eye likes to find simplicity and order in complex shapes because it prevents us from becoming overwhelmed with information.

Parallel Prototyping cont'd

•Quality of Designs: The parallel design process yielded better-performing designs (as measured both by expert critiques and click-through performance). • •Design Diversity: The parallel process generated more diverse designs. (No surprise here: a serial process implicitly encourages refinement of a single idea.) • •"Self-Efficacy" and Reactions to Critiques: Parallel participants reported an increase in task-specific self-efficacy (task-oriented confidence). Multiple alternative designs encourages investment in a creative process rather than one particular idea.

law of similarity

•The human eye tends to perceive similar elements in a design as a complete picture, shape, or group, even if those elements are separated. •Ensure that links and navigation systems are visually differentiated from normal text elements and are consistently styled.

•Distribution of sample outcomes is divided (2)

•Those likely if Ho is true •Those "very unlikely" if Ho is true

how to identify an issue

•Typically an in-person study •During a think-aloud protocol, you might observe the following: •Verbal expressions of confusion, frustration, dissatisfaction, pleasure, or surprise •Verbal of confidence or indecision about a particular action that may be right or wrong •Participants not saying or doing something that they should have done or said •Nonverbal behaviors such as facial expressions and/or eye movements • •Automated Studies •The key is to allow participants to enter verbatim comments at a page or task level. In most automated studies, several data points are collected for each task: success, time, ease of use rating and verbatim comments are the best way to understand any possible issues

metrics

•Usability is generally measured using observable + quantifiable metrics •A metric is a way of measuring/evaluating a particular phenomenon or thing. •A metric is a "system or standard of measurement" represented in units that can be utilized to describe more than one attribute. •Metrics come in very handy when it comes to quantify usability during the usability evaluation of software, websites and applications. •Common questions that come to mind when discussing metrics are: •"Which metric shall I (we) use?" •"How shall I (we) obtain the components needed to calculate it?" • "Is this metric reliable enough to give a realistic picture of the degree to which my (our) system is usable (or not)? •The field relies on a common set of metrics such as task success, user satisfaction, and errors, among others. In terms of research vernacular, metrics are referred to as dependent variables

Survey Instruments cont'd

•User background (age, gender, origins, education, income) •Experience with computers (specific applications or software packages, length of time, depth of knowledge) •Job responsibilities (decision-making influence, managerial roles, motivation) •Personality style (introvert or extrovert, risk taking or risk aversive, early or late adopter, systematic or opportunistic) •Reasons for not using an interface (inadequate services, too complex, too slow) •Familiarity with features (printing, macros, shortcuts, tutorials) •Feeling state after using an interface (confused or clear, frustrated or in-control, bored or excited)

Jakob's Law

•Users spend most of their time on other sites. •This means that users prefer your site to work the same way as all the other sites they already know. •You can simplify the learning process for users by providing familiar design patterns.

Wizard of Oz

•Wizard of Oz prototypes are prototypes with faked functions. •Interactivity that comes from a human rather than an algorithm or software code. •virtual assistant software in which a human, working on another computer, types the responses. Note: ethically, we as designers are well within the boundaries when doing this; it does not involve manipulating users for immoral gain. •This prototyping method involves a fair amount of time and effort. Therefore, you can really only make the best use of it for testing the effects and interactions of complex systems or in the later stages of your design project. •Can be done early in design to understand users' expectations.

Survey Instruments

•Written user surveys are a familiar, inexpensive and generally acceptable companion for usability tests and expert reviews •Keys to successful surveys •Clear goals in advance •Development of focused items that help attain the goals •Users could be asked for their subjective impressions about specific aspects of the interface such as the representation of: task domain objects and actions •Syntax of inputs and design of displays

eye tracking

●Area of Interest ●Gaze Replay ●Fixations ●Sequence of Fixations

Behavioral and physiological metrics

●Eye Tracking ●Measuring Emotion ●Stress and Other Physiological Measures