Human computer interaction

3D Environments

3D interaction is natural in the real-world 3D environments are common in digital games Rich graphical 3D environment are processor intensive 3D Navigation Involves two types of movement Translation - movement on a plane Rotation - movement around an axis Desktop 3D Current GUIs are predominantly 2D 3D environments presented on 2D screens are difficult to navigate Three-dimensional navigation can quickly become difficult and confusing Web-based 3D Use vector-based graphics to decrease file size Virtual Reality Modeling Language (VRML) Uses polygons with parameters Transparency Texture maps shininess X3-D is XML based - Web3D.org Offers greater flexibility and control

Direct Manipulation*

Continuous representations of the objects and actions of interest with meaningful visual metaphors. Physical actions or presses of labeled buttons instead of complex syntax. Rapid, incremental, reversible actions whose effects on the objects of interest are visible immediately. Three phases Free Phase - How the screen looks before any user actions Captive Phase - How the screen looks during a user action (click, click-drag, etc.) Termination Phase - How the screen looks after a user action Advantages: Easy to learn Low memory requirements Easy to undo Immediate feedback to user actions Enables user to use spatial cues Easy for beginners Disadvantages: Not self-explanatory Inefficient use of screen real estate High graphical system requirements

Be able to name and describe different usability principles

Effectiveness is a very general goal and refers to how good a product is at doing what it is supposed to do. Efficiency refers to the way a product supports users in carrying out their tasks. Safety involves protecting the user from dangerous conditions and undesirable situations. Utility refers to the extent to which the product provides the right kind of functionality so that users can do what they need or want to do. Learnability refers to how easy a system is to learn to use. Memorability refers to how easy a product is to remember how to use, once learned.

Expanding menus

Enables more options to be shown on a single screen than is possible with a single flat menu More flexible navigation, allowing for selection of options to be done in the same window Most popular are cascading ones primary, secondary and even tertiary menus downside is that they require precise mouse control can result in overshooting or selecting wrong options

Sample question (Excellent exam question) Two of Nielsen's heuristics deal specifically with errors. What are these heuristics?

Error prevention:Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action. Help users recognize, diagnose, and recover from errors:Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

Seven Stages of Action constitute three stages of execution, three stages of evaluation and our goals.

Forming the goal Forming the intention Specifying an action Executing the action Perceiving the state of the world Interpreting the state of the world Evaluating the outcome

Robots

Four common types remote robots used in hazardous settings domestic robots helping around the house pet robots as human companions sociable robots that work collaboratively with humans, and communicate and socialize with them - as if they were our peers Advantages Pet robots are assumed to have therapeutic qualities, being able to reduce stress and loneliness Remote robots can be controlled to investigate bombs and other dangerous materials

Brain-Computer Interfaces - BCI

provide a communication pathway between a person's brain waves and an external device, such as a cursor on a screen Person is trained to concentrate on the task, e.g. moving the cursor work through detecting changes in the neural functioning in the brain

Shareable

provide multiple inputs and sometimes allow simultaneous input by co-located groups large wall displays where people use their own pens or gestures interactive tabletops where small groups interact with information using their fingertips e.g. DiamondTouch, Smart Table and Surface Advantages Provide a large interactional space that can support flexible group working Can be used by multiple users can point to and touch information being displayed simultaneously view the interactions and have same shared point of reference as others Can support more equitable participation compared with groups using single PC

Fitt's Law

the time required to rapidly move to a target area is a function of the distance to the target and the size of the target. Closer & Larger = quicker Fitt's Equation: MT = a+b*log2(AW+1) MT is movement time (in seconds) a and b are constants A is distance to target W is width of the target a and b are constants (a = 50, b = 150 for rough estimates)

Hick's Law

describes the time it takes for a person to make a decision as a result of the possible choices he or she has:ie increasing the number of choices will increase the decision time. More choices=more time Hick's Equation: T = a + b*log2(n+1) T is time n is alternatives a and b are constants (a = 50, b = 150 for rough estimates)

Schneiderman's 8 Golden Rules for interface design

1. Strive for consistency. For example, in every screen have a 'File' menu in the top left-hand corner. For every action that results in the loss of data, ask for confirmation of the action to give users a chance to change their minds. 2. Enable frequent users to use shortcuts. For example, in most word-processing packages, users may move around the functions using menus or shortcut "quick keys," or function buttons. 3. Offer informative feedback. Instead of simply saying "Error 404," make it clear what the error means: "The URL is unknown." This feedback is also influenced by the kinds of users, since what is meaningful to a scientist may not be meaningful to a manager or an architect. 4. Design dialogs to yield closure. For example, make it clear when an action has completed successfully: "printing completed." 5. Offer error prevention and simple error handling. It is better for the user not to make any errors, i.e., for the interface to prevent users from making mistakes. However, mistakes are inevitable and the system should be forgiving about the errors made and support the user in getting back on track. 6. Permit easy reversal of actions. For example, provide an "undo" key where possible. 7. Support internal locus of control. Users feel more comfortable if they feel in control of the interaction rather than the device being in control. 8.4 Physical design: getting concrete 267 8. Reduce short-term memory load. For example, wherever possible, offer users options rather than ask them to remember information from one screen to another.

Norman's Principles

1. Visibility - The more visible functions are, the more likely users will be able to know what to do next. In Contrast, when functions are "out of sight," it makes them more difficult to find and know how to use. 2. Feedback - Feedback is about sending back information about what action has been done and what has been accomplished, allowing the person to continue with the activity. Various kinds of feedback are available for interaction design-audio, tactile, verbal, and combinations of these. 3. Constraints - The design concept of constraining refers to determining ways of restricting the kind of user interaction that can take place at a given moment. There are various ways this can be achieved. 4. Mapping - This refers to the relationship between controls and their effects in the world. Nearly all artifacts need some kind of mapping between controls and effects, whether it is a flashlight, car, power plant, or cockpit. An example of a good mapping between control and effect is the up and down arrows used to represent the up and down movement of the cursor, respectively, on a computer keyboard. 5. Consistency - This refers to designing interfaces to have similar operations and use similar elements for achieving similar tasks. In particular, a consistent interface is one that follows rules, such as using the same operation to select all objects. For example, a consistent operation is using the same input action to highlight any graphical object at the interface, such as always clicking the left mouse button. Inconsistent interfaces, on the other hand, allow exceptions to a rule. 6. Affordance - is a term used to refer to an attribute of an object that allows people to know how to use it. For example, a mouse button invites pushing (in so doing acting clicking) by the way it is physically constrained in its plastic she

Nielsen's Heuristics

A heuristic evaluation is a usability inspection method for computer software that helps to identify usability problems in the user interface (UI) design. It specifically involves evaluators examining the interface and judging its compliance with recognized usability principles (the "heuristics"). 1. Visibility of system status: The system should always keep users informed about what is going on, through appropriate feedback within reasonable time. 2. Match between system and the real world: The system should speak the user's language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order. 3. User control and freedom: Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo. 4. Consistency and standards: Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions. 5. Error prevention: Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action. 6. Recognition rather than recall: Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate. 7. Flexibility and efficiency of use: Accelerators—unseen by the novice user—may often speed up the interaction for the expert user such that the

Execution Phase

Articulation = The user formulates a goal, which is then articulated using the input language. Performance = The input language is translated into the core language (operations that the system will carry out). Presentation = The system manifests the result of the core language operations using the output language.

Be able to define common terms in HCI (e.g. direct manipulation).+

Assumptions - taking something for granted when it needs further investigation e.g. people will want to watch TV while driving Claims - stating something to be true when it is still open to question e.g. speech controlled GPS is safe Metaphor - representing an interaction in a way that exploits user's knowledge of some real life objects. Doesn't necessarily have to be entirely realistic. e.g. Desktop on a computer is meant to evoke the idea of a physical desktop. Framework - A structure that provides a context for conceptualizing something Conceptual Model - a high-level description of how a system is organized and operates Mental Models - A cognitive representation of something that defines a logical and believable estimation as to how a thing is constructed or functions. influences how a user perceives the world and tries to manipulate it Semantic Distance - The distance between what people want to do and the meaning of an interface element (similar to the difference in meanings of same words/phrases in different languages) Articulatory Distance - The distance between the physical appearance of an interface element and what it actually means (ex. difference between the meaning of long vs. looooong) Affordance Confusion - when certain aspects of an object do not work in a way which we assume they should.

Augmented Reality - AR

Augmented reality - virtual representations are superimposed on physical devices and objects Mixed reality - views of the real world are combined with views of a virtual environment Many applications including medicine, games, flying, and everyday exploring Wearable First were head/eyewear-mounted cameras Enable the recording of current events Access to digital information Jewellery, smart-fabrics, glasses, shoes, and jackets have also been used Advantages Means of interacting with digital information on the move Disadvantage Hygiene Applications Fashion Automatic diaries Tour guides

Command Line*

Commands are typed in at the prompt; the system responds Many commands are abbreviated Commands can be applied to many objects simultaneously Some commands have multiple parameters that can be set and altered Advantages: Suitable for repetitive tasks Advantageous for expert users Offer direct access to system functionality Efficient and powerful Not encumbered with graphic controls Low visual load Not taxing on system resources Disadvantages: Low command retention Steep learning curve High error rates Heavy reliance on memory Frustrating for novice users

Sample question Consider the task of whatever. Devise a KLM that represents this task. What assumptions have you made?

Consider the text editing task of searching a Microsoft Word document for all occurrences of a four-letter word, and replacing it with another four-letter word. Here we use the notation of Card et. al (1983, p264-265).

WIMP/GUI*

GUIs use visual relationships to real-world objects (metaphors) Metaphors can help people relate to complex concepts and procedures by drawing on real-world knowledge Real-world affordances can be reflected Xerox Star first WIMP -> rise to GUIs Windows could be scrolled, stretched, overlapped, opened, closed, and moved around the screen using the mouse Icons represented applications, objects, commands, and tools that were opened when clicked on Menus offering lists of options that could be scrolled through and selected Pointing device a mouse controlling the cursor as a point of entry to the windows, menus, and icons on the screen Windows Windows were invented to overcome physical constraints of a computer display enable more information to be viewed and tasks to be performed Scroll bars within windows also enable more information to be viewed Multiple windows can make it difficult to find desired one listing, iconizing, shrinking are techniques that help Icons Icons are assumed to be easier to learn and remember than commands Can be designed to be compact and variably positioned on a screen Now pervasive in every interface e.g. represent desktop objects, tools (e.g. paintbrush), applications (e.g. web browser), and operations (e.g. cut, paste, next, accept, change) Icon forms The mapping between the representation and underlying referent can be: similar (e.g., a picture of a file to represent the object file), analogical (e.g., a picture of a pair of scissors to represent 'cut') arbitrary (e.g., the use of an X to represent 'delete') Most effective icons are similar ones Many operations are actions making it more difficult to represent them use a combination of objects and symbols that capture the salient part of an action

Sample question An interface must provide users with a list of ten items from which they can choose one or more. Provide two alternative designs for this functionality and identify the strengths and weaknesses of each.

List boxes and check boxes

Be able to use the vocabulary of windowing systems correctly and be able to perform small design tasks on the fly.

Maximized: window occupies whole screen Minimized: reduced to a button or icon Restored: returns to previous dimensions, becomes resizable, and can overlap other windows Floating toolbox Floating dialog Application window Desktop Task bar Multiple Document Interface (MDI) - Workspace for all open documents, only one primary window. Adv: Conserve system resources, only one instance of application. create minimal visual clutter. (e.g. Photoshop window with multiple photos open) Single Document Interface (SDI)- open new primary windows for each instance of an application Tabbed Document Interface (TDI)/(Workbook) Modal/Modeless Dialogues Expanding Dialogues Anything with Details >> or Advanced >> Panes, Frames, and tabs Icons Menus Menu Bar Pull-down menu Cascading menu Pop-ups Radio Buttons Checkboxes Pointers Listbox Dropdown to select things from Combobox Listbox that you can type in to search Spinner Usually a number field with up or down arrows to increment or decrement Slider A slider to edit the value Command Buttons Submit, Reset, etc. Hyperlinks Scrollbars Splitters Text Box Text Field

Multi-modal

Meant to provide enriched and complex user experiences multiplying how information is experienced using different modalities, i.e. touch, sight, sound, speech support more flexible, efficient, and expressive means of human-computer interaction Most common is speech and vision

Be able to analyze an interface or device in terms of its usability.

Memorability - Can a user remember enough to use it effectively the next time? Ease of learning - how fast can user learn to accomplish basic tasks Efficiency of use- How fast can an experienced user accomplish tasks? Subjective satisfaction: how much does the user like using the system? Error frequency and severity - How often do users make errors, how serious are these errors, and how do users recover from these errors? "MEESE"

Natural Language*

Natural Language Interaction (NLI) - Interacting with computers using everyday language Advantages of NLI: Ease of learning Low memory requirements Flexible interaction Low screen requirements Appropriate for beginners Disadvantages of NLI: Requires knowledge of the task domain May require tedious clarification dialogues Complex system development

Gesture

Now commonplace in games and on some specialized platforms. Uses camera recognition, sensor and computer vision techniques can recognize people's body, arm and hand gestures in a room systems include Kinect and EyeToy Movements are mapped onto a variety of gaming motions Players represented on the screen as avatars doing same actions Advantages: Some gestures are natural Natural gestures can disambiguate commands Disadvantages: Some gestures can be tiring Gestures may not be easy to learn Gestures may not be easy to explain The ability of a system to respond to specific gestures is often invisible

Touch

Now commonplace on mobile devices like smartphones and tablets. Fingers are used as a pointing device. Advantages: [Direct manipulation] No extra pointing device needed Multitouch is more versatile than conventional devices Disadvantages: Fingers are large and relatively imprecise Fingers and hands can obscure the display Accidental touches are common The ability of a system to respond to specific touch gestures is often invisible in the display

Evaluation Phase

Observation = The user interprets the results on the screen and reconciles them with the original goal.

KLM operators

Operator Description Time (s) K represents pressing a key or a button 0.2 P Point with mouse to a target on the display 1.10 H Home hand(s) on keyboard or other device 0.40 M Mentally prepare 1.35 R(t) Response time (t) by system t

Menus*

Presents users with sequential hierarchical menus that offer lists of functions. Textual: key-in number of option Graphical: use arrow keys or pointing device Menus are based on recognition as opposed to recall No need to remember commands Users search from a list of possible choices List provides constraints Appropriate for small screens (iPod) Advantages: Low memory requirements Self-explanatory Easy to undo errors Appropriate for beginners Disadvantages: Rigid and inflexible navigation Inefficient for large menu navigation Inefficient use of screen real estate Slow for expert users

Contextual menus

Provide access to often-used commands that make sense in the context of a current task Appear when the user presses the Control key while clicking on an interface element e.g., clicking on a photo in a website together with holding down the Control key results in options 'open it in a new window,' 'save it,' or 'copy it' Helps overcome some of the navigation problems associated with cascading menus

Question and Answer

Question and answer interfaces are also called wizards. They are restricting for expert users They are easy for novice users However, they may not know the required information Advantages of question and answer interfaces: Low memory requirements Self-explanatory Simple linear presentation Easy for beginners Disadvantages of question and answer interfaces: Require valid input supplied by user Require familiarity with interface controls Can be tedious to correct mistakes

Form Fill-In

Similar to menu interfaces present screens of information Different than menu interfaces - used to capture information and proceed linearly not to navigate a hierarchical structure Forms can be presented using Single scrolling screen Multiple linked pages Form elements must be grouped logically Include "You Are Here" indications Inform the user about the length and where they are within the structure Form elements must be unambiguously labeled to increase data integrity Users must understand what data is required and what format should be used Date information formats: 1/29/2005, 29/1/2005, or January 29, 2005? Advantages: Low memory requirements Self-explanatory Can gather a great deal of information in little space Present a context for input information Disadvantages: Require valid input in a valid format Require familiarity with interface controls Can be tedious to correct mistakes

Haptic

Tactile feedback applying vibration and forces to a person's body, using actuators that are embedded in their clothing or a device they are carrying, such as a cell phone Can enrich user experience or nudge them to correct error Can also be used to simulate the sense of touch between remote people who want to communicate

KLM: Heuristics for M placement

The KLM operators can be placed into one of two groups—physical or cognitive. The physical operators are defined by the chosen method of operation, such as clicking an icon or entering a command string. The cognitive operators are governed by the set of heuristics for placing Ms. Rule 1: Insert Ms before all Ks. Insert Ms before Ps that select commands (but not arguments). Rule 2: If M is fully anticipated, delete M (e.g. PMK— PK). Rule 3: If MKs are a unit, delete Ms after the first (e.g. MKMKMKMK—MKKKKK). Rule 4: If K is a redundant delimiter, delete M before it (e.g. return or space after argument). EX. MPMK[Click]MK[Return] -R2-> MPK[Click]MK[Return] -R4-> MPK[Click]K[Return] Rule 5: Delete Ms that overlap with Rs.

Be able to describe the general characteristics of human cognitive processing, structurally and procedurally.

The Working Memory (WM) must interact with Long Term Memory (LTM) over a significant length of time before an item can be stored in LTM. This increases the number of cues that can be used to retrieve the item later Items with numerous associations have a greater probability of being retrieved Cognitive—The cognitive system bridges the perceptual and motor systems It can function as a simple conduit or it can involve complex processes, such as learning, fact retrieval, and problem solving Cognitive coding in the WM is predominantly visual and auditory

Interaction Styles or Techniques

This Covers Two Sections of the Study Guide Be able to describe different interaction techniques, including their strengths and weaknesses. (page 158) For each of the interaction styles covered in the book, be able to describe it, to give go examples of guidelines in building interfaces using the style, and to summarize its advantages and disadvantages (i.e., situations in which a given style is appropriate or inappropriate, based on the characteristics of the task and the user.) (chapter6-interaction-techniques slides) Remember: Common Types of Interaction - A "type" may be a "style" or a "technique"

Web Navigation*

Two basic interaction Type Link-based navigation Sensitive to articulatory distance Ambiguous link labels increase the gulf of evaluation Search Sensitive to semantic distance Inadequate search engine algorithms increase the gulf of execution Slight advantage in development of mental models

Tangible*

Type of sensor-based interaction, where physical objects, e.g., bricks, are coupled with digital representations When a person manipulates the physical object/s it causes a digital effect to occur, e.g. an animation Digital effects can take place in a number of media and places or can be embedded in the physical object Advantages Can be held in both hands and combined and manipulated allows for more than one person to explore the interface together encourages different ways of representing and exploring a problem space People are able to see and understand situations differently can lead to greater insight, learning, and problem-solving can facilitate creativity and reflection

Be able to produce a standard formal notation (e.g. a state machine) for a given scenario. Be able to decide on appropriate abstractions and simplifications, if necessary.

Universal Design: Designing all products and the built environment to be aesthetic and usable to the greatest extent possible by everyone.Hearing loss, eyesight, movement troubles, speed, age, culture, etc. Principles of Universal Design: • Equitable use • Flexibility in use • Simple and intuitive to use • Provide perceptible information • Tolerance for error • Low physical effort • Size and space for approach and use

Be able to define the term usability, as discussed in class.

Usability is the ease of use and learnability of a human-made object. The object of use can be a software application, website, book, tool, machine, process, or anything a human interacts with. Usability Definition he gave in class: The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. Usability Definition from section 1.6.1: Usability refers to ensuring that interactive products are easy to learn, effective to use, and enjoyable from the user's perspective

Design Principles (section 1.6.3) [same as Norman's Principles]

Visibility: make all needed options and materials for a given task visible without distracting the user with extraneous or redundant information. Example: Consider all the buttons clearly labeled in a car versus an automated soap dispenser Feedback: keep users informed of actions that have completed and what has been accomplished Example: Consider playing the guitar where the sounds were on 5 second delay, not having immediate feedback that you had actually struck a note would be frustrating Constraints: make the user aware of what they can/cannot do at a given moment Example: Consider GUIs with menu items that are grayed out until certain conditions are met (in a text editor: copy being grayed out until text is selected) Physical - not possible to move cursor off screen logical - you scroll DOWN to move the page DOWN cultural - scroll bar being to the right because it has always been that way Mapping: This refers to the relationship between controls and their effects in the world. Example: Cut icon → cut action Consistency: similar operations using similar elements for achieving similar tasks Example: In your text editor being able to highlight text using left mouse button and in your browser only by using right mouse button Affordance: Objects in an interface should have perceivable action possibilities Example: Mouse buttons invite users to click them, and drop-down arrows in web browser suggest a menu.

Sample question What are the stages in the EEAC for some situation? What are the gulf of evaluation and the gulf of execution?

What are the stages in the EEAC for some situation? What are the gulf of evaluation and the gulf of execution? EEAC Stages Form the goal Form the intention Specify an action Execute the action Perceive the state of the world Interpret the state of the world Evaluate the outcome Gulf of execution: difference between intentions and allowable actions Gulf of evaluation: the amount of effort that the person must exert to interpret a system state, and determine how well expectations and intentions have been met

Speech

Where a person talks with a system that has a spoken language application, e.g., timetable, travel planner Used most for inquiring about very specific information, e.g. flight times or to perform a transaction, e.g. buy a ticket Also used by people with disabilities e.g. speech recognition word processors, page scanners, web readers, home control systems FORMAT Directed dialogs are where the system is in control of the conversation Ask specific questions and require specific responses More flexible systems allow the user to take the initiative: e.g. "I'd like to go to Paris next Monday for two weeks." More chance of error, since caller might assume that the system is like a human Guided prompts can help callers back on track e.g. "Sorry I did not get all that. Did you say you wanted to fly next Monday?"