Methods for Usability Evaluation

Results of GOMS analysis can be used to predict _____________ ___________.

human learning

USABILITY TESTING

- 1. Focus is on usability - 2. Participants are end users or potential users - 3. Participants can be asked to think aloud as they do the task - 4. Data are recorded and analyzed.

Collecting Data for Usability Testing:

-You can use performance measures, -subjective measures, -and video recordings.

Questionnaires

-users' subjective evaluations of a product or software program. - includes a 5-point or 7-point scale, which can be used to measure how good or bad the user finds the situation. -Questionnaire must also be tested

5 KEY FEATURES OF COGNITIVE WALKTHROUGH

- 1. Performed by analyst and reflects analyst's judgement. - Not based on data from test users - 2. Examines specific user tasks, rather than analyzing global aspects of user interface. - 3. Analyst investigates correct sequences of users' actions, asking if user can perform these sequences using the intended interface design. - 4. Identify likely trouble spots in an interface and suggest a reason for the trouble - 5. Analyst identifies problems by tracing the mental processes of the user, not by focusing on the interface itself. - Analyst has to understand the background knowledge of users and try to infer how to influence or change mental processes.

COGNITIVE WALKTHROUGH:

- 1. Performed by analyst and reflects analyst's judgement. - Not based on data from test users - 2. Examines specific user tasks, rather than analyzing global aspects of user interface. - 3. Analyst investigates correct sequences of users' actions, asking if user can perform these sequences using the intended interface design. - 4. Identify likely trouble spots in an interface and suggest a reason for the trouble - 5. Analyst identifies problems by tracing the mental processes of the user, not by focusing on the interface itself. - Analyst has to understand the background knowledge of users and try to infer how to influence or change mental processes.

4-POINT SEVERITY SCALE:

- 1. Preventing completion of a task, such as consistently picking the wrong menu option and not knowing what to do - 2. Significant delay and frustration. Ex: no feedback to confirm the action user has taken - 3. Minor effect on usability, like using the same word for 2 diff actions. - 4. Enhancements that can be added in future, like user request for tutorial to comment on a few issues.

Comparison of Usability Methodologies - Broken down into 3 categories:

- 1. Skill-based performance: errors committed are because of problems in perception and/or motor response - 2. Rule-based performance: Error committed are related to selection of the wrong procedure, and often lead to omission of steps in a procedure. - 3. Knowledge-based performance: Errors are related to user's lack of knowledge. Users perform incorrect actions, bcs they don't understand the problem.

HEURISTICS EVALUATION:

- 1. consistency - 2. shortcuts - 3. informative feedback - 4. Design dialogues to yield closure - 5. error prevention and simple error handling - 6. reversal of actions - 7. Support internal locus of control (user in control of the system) - 8. Reduce short-term memory load

- Advantages n disadvantages with the diff methods - COGNITIVE WALKTHROUGH:

- Advantage: can analyze user's mental processes and can be used to understand how lack of user knowledge and experience can make an interface hard to understand.

User Participatory Design:

- Arguments in favor of participatory design: - Accurate info about tasks - Democratic decision making - Sense of participation - Potential for better user acceptance - Arguments against participatory design: - Takes long time - Forces designer to comprise design goals - Users don't understand Human-Computer interaction - Users don't understand negative effects of poor design.

Advantages n disadvantages with the diff methods - HEURISTICS EVALUATION:

- Evaluators don't make any assumptions about user knowledge, & therefore can't analyze reasons behind user errors.

Analytical evaluations: 3 common methods

- Heuristics evaluation - Cognitive walkthrough - Model-based analysis such as goals, operators, methods, and selection (GOMS)

Many traditional methods available for doing this:

- Task analysis - Interviews with users - Observations of users using similar systems - Intuition on the part of the analyst

One way of analyzing the data

- To measure efficiency: use performance time - time it takes to perform a task - Effectiveness: task success. To what extent did user manage to accomplish what they were supposed to accomplish? - Satisfaction: measures related to user's ratings and verbal comments.

- User-based evaluations: testing by users

- Usability testing - User participatory design Thinking aloud

2 main categories of usability methods: Analytical evaluations:

- also called usability inspection methods - Product is evaluated by a professional analyst

HEURISTICS EVALUATION:

- design goals that are used to evaluate an interface

Performance measures:

- involve the time to perform a task, -# of errors, types of errors, -# of times experimenter had to step in to help the test person.

Advantages n disadvantages with the diff methods - USER TESTING

Advantage: face validity

Advantages n disadvantages with the diff methods - THINK-ALOUD METHOD

Advantage: gives insights to the users reason, make conclusions, and commit errors. Can be efficient in removing causes for error.

Advantages n disadvantages with the diff methods - GOMS

Disadvantage: can't analyze errors

Goals, Operators, Methods, and Selection (GOMS) should be (GMOS)

Goal: Specification of what user wants to achieve. Operators: Actions that analyzed software allows user to take Method: methods that can be used to accomplish a goal. Selection: Rules that user follows in deciding which method to use.

Great the number of steps . . . . . . . . . .

Longer the learning time

- GOMS does not identify _________ _________. The _________ has to identify the goals.

user's goals. user

Think-Aloud Protocols

users talk out loud their thinking process while interacting with the product. Purpose : you can see the users intentions and actions Problem: speaking adds to the workload and they participant may perform worse.