UI

Temporary Impairment

Impairments that improve over time, such as recovery after illness or accidents, e.g., a broken arm.

Elements of a Posture-level Pattern

Once we determine the posture of an application, it gives us guidance on: » Structure » Components » User experience » Alternatives/competitors

Conversational Interfaces

Interfaces where the interaction with the system has the characteristics of human conversations, including turn taking, theoretically infinite depth, conversational markers, etc.

Usability Heuristics for CIs

1. General -Give the agent a persona through language, sounds, and other styles -Make the system status clear -Speak the user's language -Start and stop conversations -Pay attention to what the user said and respect the user's context 2. Conversational style -Use spoken language characteristics -Make conversation a back-and-forth exchange -Adapt agent style to who users are, how they speak, and how they are feeling 3. Guiding, Teaching, and Offering Help -Guide users through a conversation so they are not easily lost -Use responses to help users discover what is possible 4. Feedback and prompts -Keep feedback and prompts short -Confirm input intelligently -Use speech-recognition system confidence to drive feedback style -Use multimodal feedback when available 5. Errors -Avoid cascading correction erros -Use normal language in communicating errors -Allow users to exit from errors or a mistaken conversation

Types of Errors (CIs)

1. No speech detected 2. Speech detected, but nothing recognized 3. Something was recognized correctly, but the system does the wrong thing with it 4. Something was recognized incorrectly

Confirmations

CIs are designed with explicit forms of confirmation to improve system usability and transparency. Can be explicit vs. implicit and speech-based vs. non-speech based (visual, action). Explicit con!rmation: Requiring the user to confirm: "I think you want to set a reminder to 'buy insirance before going skydiving next week.' Is that right?" Implicit con!rmation: Letting user know what was understood: "Ok, setting a reminder to buy insurance..."

Dimension 2: Look and Feel (3D Model of Prototyping)

Simulates the sensory experience of the user while using the system, i.e., what the user sees, hears, and feels during use.

Turn-talking

Speaking turns is the core, cooperative structure of conversations that involves one speaker at a time and an explicit exchange of tokens. Principles: 1. One speaker at a time — transparency in who is speaking 2. Turn exchanges — explicit signaling of what will speak next 3. Interruption handling — very difficult with CIs

Conversational Markers

Speech cues that indicate the state or the direction of the conversation. Types of conversational markers: 1. Timelines ("First," "Halfway there," "Finally") 2. Acknowledgements ("Thanks," "Got it,", "Alright," "Sorry about that") 3. Positive feedback ("Good job," "Nice to hear that")

React Native Accessibility Actions

Standard, e.g., magicTap, escape, activate, increment, decrement, longpress, or custom actions, handled by onAccessibilityAction.

Design Languages

A vocabulary of design elements that are repeatedly applied to interaction design problems. UX Design Languages: Task- and action-level interface components that follow a consistent look and feel in appearance and behavior.

Prototyping

Building a draft or an early version of a product or system in order to explore, demonstrate, and test design ideas for a part or the entirety of the product or system.

Incremental Prototyping (Prototyping Strategies)

Dividing system functionality into slices (vertical prototypes) based on design specifications and incrementally building each slice that is then integrated into the overall system. APPROPRIATE FOR LARGE AND COMPLEX PROJECTS.

Multi-touch Gestures

Mobile input method include a number of idiomatic gestures dedicated to specific functions.

Panes and Panels (Mobile Design Patterns)

Multi-pane structures and pop-up panels and tools are commonly used in tablets to provide secondary application windows in a way that's similar to desktop applications

Prototyping Models & Strategies

» Three-dimensional model of prototyping » Prototyping scope » Prototyping strategies » Prototyping fidelity

Steps of Experience Prototyping

1. Define context (user interaction with a clothing retail website) 2. Develop scenarios (e.g., find products, read reviews) 3. Identify design goals (how should the user be supported?) 4. Set up environment (very minimal in this design problem) 5. Act out interaction (assign roles, bodystorm) 6. Develop insight (take notes on what you learned)

Criteria for choosing right prototype method

Design team goals, capabilities and resources, and available time for prototyping

Native Prototyping

Implementing and testing design ideas on the target technology platform of the design.

Feedback (Microinteractions)

Information that the user sees, hears, or feels are forms of system feedback on what is happening.

Command-and-Control Interfaces

Interfaces where speech input is mapped to specific system functions that are called immediately. These interfaces commonly utilize: 1. Expressing user intent using a wake word (e.g.,"OK, Google") or the pressing of a button (e.g., home button in the iPhone) 2. Indicating listening and understanding 3. Executing the mapped function

Annotations (Prototyping Method)

Labels, explanations, and notes that provide further information on the goals, content, and functioning of the design elements illustrated on wireframes. KEY IN ADDRESSING THE PROBLEM OF INTERPRETABILITY OF SIMPLIFIED DESIGNS FOR ALL STAKEHOLDERS.

Paper Prototyping

MOST USEFUL AT THE EARLIEST STAGES OF DESIGN PROCESS. Mocking up design ideas by sketching pages/screens with design elements using design supplies (e.g., paper, pencils, markers, scissors, glue, tape) and simulating how the envisioned system would respond to user input by swapping different pages/ screens and moving/changing design elements.

Loops & Modes (Microinteractions)

Meta-rules that, depending on context, change microinteraction rules and feedback (e.g., "snoozing" a reminder; chime/vibration feedback when silent mode is off/on). Loops determine the length of the micro interaction and whether the interaction repeats (e.g., related beeping when you leave the fridge door open) or changes over time (e.g., microwave over reminder to pick up food changing over time). Modes switch the functioning or operation of the system. E.g., "do not disturb" mode that changes system behavior.

Multimodality (CI DP)

Multimodal interfaces utilize multiple modalities, including visual information, speech, touch, and so on, in user experiences they afford. Most conversational interfaces are multimodal interfaces. Multimodality Principle: Take advantage of other modalities, e.g., visual information, vibrations, etc., wherever appropriate. Using multimodal components, you can provide users with breaks for decision making, interruptions, etc. Potential caveats: » Ask, "does my interface still support a speech-only interaction?" » The conversational and other components must be designed together to fit within the conversation.

Principles of Universal Design

1. Equitable use 2. Flexibility in use 3. Simple and intuitive use 4. Perceptible information 5. Tolerance for error 6. Low physical effort 7. Size and space for approach and use

Three Dimensions of Disability

1. Impairment in a person's body structure or function, or mental functioning (e.g., loss of a limb, loss of vision, or memory loss) 2. Limitation in activities (e.g., difficulty seeing, hearing, walking, or problem solving) 3. Restrictions in participation in activities of daily living (e.g., working, engaging in social and recreational activities, and obtaining health care)

Where are conversational interfaces valuable?

1. Streamlining app installation, login, payment, notifications, and and so on in a conversational paradigm.1 2. In some contexts, e.g., while driving, CIs are more effective, efficient, and satisfactory due to resource constraints. 3. CIs address many accessibility problems, including vision (e.g., blindness), motor (e.g., tremor), and cognitive (e.g, dyslexia) deficiencies.

Pattern Languages

A complete and hierarchical collection of patterns for a family of design problems. Patterns are words (e.g., a component) that are connected with grammar rules to make sentences (e.g., a screen) and eventually language (e.g., user experience). The pattern language can be thought of as patterns being applied at different levels.

Evolutionary Prototyping/Breadboard Prototyping (Prototyping Strategies)

Also called breadboard prototyping, the design team incrementally builds prototypes of a design idea, tests the idea with users, and refines the prototype until it reaches the desired level of maturity. Most products we use are evolutionary prototypes that went through alpha, beta, etc. phases

Advanced Direct Manipulation (Mobile Design Patterns)

Applications, such as image editors, drawing or presentation tools, or media players, enable direct-manipulation-based controls for content creation or editing. These patterns provide customized user-input methods, such as control handles for object manipulation or scrollbars to navigate through video content

Size and Space for Approach and Use (Princ of Uni Design)

Appropriate size and space is provided for approach, reach, manipulation, and use regardless of user's body size, posture, or mobility. 1. Provide a clear line of sight to important elements for any seated or standing user. 2. Make reach to all components comfortable for any seated or standing user. 3. Accommodate variations in hand and grip size.

Extreme Prototyping (Prototyping Strategies)

Breaking down the development into three phases that build on each other: (1) building a static prototype, (2) building fully functional, interactive components that will simulate services, and (3) finally implementing the services. Enables rapid and parallel prototyping, testing, and refinement by removing dependencies between different components of a system or between the system and third party services

Card (Mobile Design Patterns)

Cards are similar to lists and grids, but they put together different compositions of multimedia content, including images, text, and links, on a column, row, or grid that users navigate through by swiping horizontally and vertically.

Carousels (Mobile Design Patterns)

Content carousels provide a row of content items including images or textual cards that users can navigate through by swiping left and right.

Interaction Paradigm (CI DP)

Conversational interfaces can follow different paradigms depending on the context of use and the design of the application: » Command-and-control interfaces (always-on voice assistants) » Conversational interfaces (chatbots, task assistants, social robots)

Interactive Prototyping

Creating realistic prototypes of the navigation or structural components (or both) of the design idea by creating a series of screens/pages with design elements, linking these screens/pages for navigation, and simulating the transitions between screens/pages. CAN USE LO-FI OR HI-FI COMPONENTS.

Task-Level Patterns

Design solutions that help users accomplish sequences of actions that make up user tasks, e.g., logging a meal, capturing a run, or completing a workout. Tasks point to specific application components. E.g., meal logging can be done through a "search-and-filter" component, activity tracking can be done through a "scoreboard" component. Task-level patterns can be domain independent. Business goals and posture-level patterns set the context for these patterns.

Direct Manipulation Input

In web and desktop interfaces, direct manipulation input involves mouse or trackpad input that is mapped to the screen using a relative mapping. On mobile devices, this mapping is absolute, and the user directly interacts with screen elements.

Landing Pages, Guided Tours (Mobile Design Patterns)

Includes a landing, welcome, or home screen that can serve as a portal to the most frequently used functions or as a guide to the next action in the task. This pattern can also serve as guided tours for new users or serve as help screens by overlaying instructions or tooltips on the screen.

Lists & Grids (Mobile Design Patterns)

Lists involve vertically stacking a large number of items, including text, controls, and thumbnails, and supporting navigation through vertical scrolling. Grids involve a large continuous grid or multiple panes of grids that users can scroll through vertically or horizontally. Grids are more commonly used for information with more visual content, such as media thumbnails, icons, and photos.

Wireframes (Prototyping Method)

MOST USEFUL IN THE EARLY-TO-MID STAGES OF THE DESIGN PROCESS. Lo-fi prototypes of the makeup of a design in terms of its structural components. Wireframes can be hand-drawn or digitally created.

Gestures: Three-finger Pinch and Spread

Mapped to OS-level actions, such as copy, cut, and paste, that can usually be customized.1

Gestures: Tap

Maps to a "click" on the desktop computer to select objects or to activate/toggle the state of a control.

Gricean Maxims (CI DP)

Proposed by Paul Grice, conversational follow the cooperative principle and four key maxims: » Maxim of quality (truthful and accurate communication) » Maxim of quantity (just the right amount of information) » Maxim of relevance (appropriate and relevant information) » Maxim of manner (clear, cooperative communication)

Three-dimensional Model of Prototyping

Prototypes represent three dimensions of a design idea: 1. Role 2. Look and feel 3. Implementation Each dimension can be represented at various levels of fidelity. Their integration makes a working prototype or a pre-alpha product.

Throwaway Prototyping (Prototyping Strategies)

Rapid prototyping to explore design ideas, demonstrate feasibility, communicate with stakeholders, and test the ideas with users and eventually discarding the prototype instead of further developing the model into a final product. MOST LO-FI AND PAPER PROTOTYPES ARE THROWAWAY PROTOTYPES. USUALLY COMBINED WITH SKETCHING.

Dimensions Combined: Integration (3D Model of Prototyping)

Represents the complete user experience with the system as envisioned in the conceptual design.

Dimension 1: Role (3D Model of Prototyping)

Represents the functions that the system serves in the user's life, i.e., how the system is useful to them.

Rules (Microinteractions)

Rules determine what happens (and doesn't happen) in the system when a microinteraction is triggered.

Assistive Technologies

Screen Readers, Screen Magnification, Text Readers, Braille for the Web, Alternative Input devices,

Low Physical Effort (Princ of Uni Design)

The design can be used e!ciently and comfortably and with a minimum of fatigue. » 1. Allow user to maintain a neutral body position. 1. Use reasonable operating forces. 2. Minimize repetitive actions. 3. Minimize sustained physical effort

Perceptible Information (Princ of Uni Design)

The design communicates necessary information e!ectively to the user, regardless of ambient conditions or the user's sensory abilities. 1. Use different modes (pictorial, verbal, tactile) for redundant presentation of essential information. 2. Provide adequate contrast between essential information and its surroundings. 3. Maximize "legibility" of essential information

Equitable Use (Princ of Uni Design)

The design is useful and marketable to people with diverse abilities. 1. Provide the same means of use for all users: identical whenever possible; equivalent when not. 2. Avoid segregating or stigmatizing any users. 3. Provisions for privacy, security, and safety should be equally available to all users. 4. Make the design appealing to all users.

Tolerance for Error (Princ of Uni Design)

The design minimizes hazards and the adverse consequences of accidental or unintended actions. 1. Arrange elements to minimize hazards and errors: most used elements, most accessible; hazardous elements eliminated, isolated, or shielded. 2. Provide warnings of hazards and errors. 3. Provide fail safe features. 4. Discourage unconscious action in tasks that require vigilance.

Why is fidelity important?

The more "done" the prototype looks, the narrower the feedback will be; and vice versa. As a principle, use no higher resolution than is necessary

Posture-Level Patterns

The structure that an application follows, i.e., what type of application it is, e.g., "a calorie tracking app," "a a step counter app," or "a life coaching app."

Bars (Mobile Design Patterns)

These are vertical or horizontal strips containing buttons or tabs dedicated to different components or tools. 1. Tab bars that are placed at the top, bottom, or the side of the screen enable navigation between different components. 2. Bars can also serve as toolbars to activate various application or operating-system-level functions. 3. Bars help navigation by linking to previously viewed content or to view the previous/ next item among multiple screens.

When do we do experience prototyping?

Three key uses: 1. Understanding existing user experiences and context 2. Exploring and evaluating design ideas 3. Communicating ideas to an audience

Gestures: Drag/Swipe

Used to scroll through content, move objects, or adjust controls. These gestures are the most commonly used gestures on mobile devices. Drag involves a swipe and hold at the same time. In scrolling, dragging vertically scrolls through large amounts of content, and dragging horizontally scrolls through carousels, opens drawers, and navigates to the previous/next screen. Dragging can also move objects, such as shapes on a canvas, or operate controls, such as knobs, switches, sliders, and virtual control pads.

Stacks (Mobile Design Patterns)

Used to vertically organize design elements such as a toolbar, content panes, and a navigation bar to maximally utilize the vertical space in mobile devices.

Why lo-fi prototyping?

» Has lower development cost » Prevents designers from prematurely wedding to specific design ideas » Enables exploring, communicating, and testing of conceptual designs » Helps designers identify structural, navigation, and organizational issues » Allows rapid evaluation of multiple design ideas » Enables communication among stakeholders » Allows identifying market requirements before dedicating resources to development

Limitations of Lo-fi Prototyping

» Requires a facilitator to drive the prototype during testing and communication » Offers limited ability to identify breakdowns in design » Lacks sufficiently low-level specifications for development » Provides limited sense of feasibility

Action-Level Patterns

Design solutions that support the actions taken to complete the steps(s) of the user's task, e.g., a "start" button to initiate activity tracking, a selectable list entry for a food item. Action-level patterns are the lowest level of building blocks for a design. They are often called widgets or components (as in React).

Drawers (Mobile Design Patterns)

Drawers provide links for navigation or controls for the various settings of the application.

Microinteractions

Microinteractions are contained product moments that revolve around a single use case. E.g., Facebook like button, pull-to-refresh action, flicking a notification to dismiss it, etc. Pro Tips: Minimize the scope of the microinteraction and make sure that it doesn't turn into a feature. Best microinteractions support single task, single action. Simpler way of thinking about microinteractions is ACTION-FEEDBACK PAIRS FOR A SINGLE PURPOSE.

Horizontal Prototype (Prototyping Scope)

Provides a broad view of the entire system and focus on the user interaction rather than the functionality.

Gestures: Two-finger Pinch and Spread

Used to shrink or expand visual elements, e.g., changing the scale of a map.

Three-levels of patterns:

1. Context: Type of app 2. Flow: Components that support specific functions 3. Implementation: The visual/ behavioral elements that implement the functions

What is it that we prototype? (Experience Prototyping)

1. System behavior 2. User behavior 3. Interactions with context

Bodystorming (Experience Prototyping)

Bodystorming is a creativity method that involves physically experiencing a situation to develop new ideas and insights.

Accessibility in React Native

RN provides us with access to assistive technologies that mobile platforms provide (e.g., VoiceOver on iOS or TalkBack on Android) through component attributes. accessible attribute indicates whether the component is an accessibility element and, if so, groups its children in a single selectable component. accessibilityLabel attribute defines screen reader descriptions of components. accessibilityHint attribute helps users understand what will happen if they perform the action on the accessibility element

Swimlanes (Mobile Design Patterns)

Swimlanes are stacked content carousels that each show a row of items, enabling visual browsing through several different lists with minimal navigation.

Flexibility in Use (Princ of Uni Design)

The design accommodates a wide range of individual preferences and abilities. 1. Provide choice in methods of use. 2. Accommodate right- or left-handed access and use. 3. Facilitate the user's accuracy and precision. 4. Provide adaptability to the user's pace.

Fidelity in Prototyping

The level of detail in which a design is represented in the prototype. » Two ends of a fidelity spectrum: » Low-fidelity (lo-fi) prototyping » High-fidelity (hi-fi) prototyping

Motion Gestures

Gestures that involve moving the mobile device in specific ways, e.g., shaking to enable/disable silent mode. These gestures are usually application specific or customizable at the OS level.

Situational Impairment

Impairments introduced by context, such as environments with low light or noise. (New parent, distracted driver, bartender, heavy accent)

Gestures: Triple-swipe

Mapped to specific OS-level activities, e.g., undo, that can usually be customized.

Mobile Design Patterns

Mobile platforms are highly constrained design environments. Mobile design patterns help designers overcome these limitations by expanding capabilities for input, display, and navigation.

What is the problem with design patterns?

Problem 1. Can I piece together different patterns to make a complete design? No, as this eclectic design would lack coherence. Problem 2. How do I choose which pattern to use? Are patterns interchangeable? No, there has to be a principle to the selection of patterns

Disability

A disability is any condition of the body or mind (impairment) that makes it more difficult for the person with the condition to do certain activities (activity limitation) and interact with the world around them (participation restrictions). Disability can change over time with age or recovery, and the severity of the impact of disability can change over time. Fewer than 20% are born with a disability, although 80% of people will have a disability once they reach 85.

Universal Design

The design of products and environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design.

Search, Sorting, and Filtering (Mobile Design Patterns)

Used to enable search and filtering to navigate through a large body of content that may be distributed across the entire navigation structure of the application. These patterns usually provide a search box to enter a search query either by typing text, voice input, or selecting from among a history of searches.

AccessAccessible Rich Internet Applications (ARIA)

aria is a set of HTML attributes that make web components available to assistive technologies.

Elements of an Experience-Level Pattern

» Primary goals, e.g., activity tracking » Secondary goals, e.g., community building

Prototyping Strategies

» Throwaway prototyping » Evolutionary prototyping » Incremental prototyping » Extreme prototyping

Types of Impairment: Anatomical

1. Sensory impairment 2. Physical impairment 3. Cognitive impairment

Design Patterns

A design pattern is a general, reusable solution to a commonly occurring problem within a given context. In UX: In the last decade, designers have also developed and refined patterns for overall structure and organization, components and controls.

How do we use pattern languages?

Common practice: Patterns in the higher levels are defined informally, and the task- and action-level patterns are adopted through experimentation and trial and error. The problem: Ineffective (e.g., lack of coherence across different levels) and inefficient (wasted effort in experimentation). The solution: Defining patterns top to bottom will "generate" the design when patterns are available across all levels.

Business Goals

Conceptual design that captures the role that the design plays in user's life, i.e., the mission of the application, e.g., "helping users achieve fitness goals."

Permanent Impairment

Congenital or long-term conditions, such as color blindness, missing body parts, etc.

Prototyping Scope

Consider the space of features and functioning as everything that a system does.

Trigger (Microinteractions)

Events that initiate the microinteractions. Triggers can be manual/user-initiated for automatic/system-initiated. Manual triggers involve intentional and explicit interaction with the system, e.g., by flipping a switch, pressing a button, speaking to the system. Automatic triggers occur when certain conditions are met, e.g., chime when a new text message arrives, swoosh when an email is sent.

Vertical Prototype (Prototyping Scope)

Focuses on a single feature/functionality and provides the full functioning of that feature.

Screen Carousels (Mobile Design Patterns)

Full-screen content panes that can be placed on a horizontal array to display different instances of the same kind of information, such as weather information for different cities.

Dimension 3: Implementation (3D Model of Prototyping)

Includes the technical capabilities that enable the system to perform its function, i.e., the low-level details of how the system works.

Gestures: Long-press

Opening contextual menus, previews of content, or enabling editing modes.

Prototyping Methods

Paper prototyping Wireframes Annotations Interactive prototyping Native prototyping

Experience Prototyping

Prototyping the holistic experience of interacting with a product. An experience prototype is any kind of representation, in any medium, that is designed to understand, explore or communicate what it might be like to engage with the product, space or system we are designing.

Usability

The effectiveness, efficiency, and satisfaction with which a specified set of users can achieve a specified set of tasks in a particular environment.

Accessibility

The usability of a product, service, environment, or facility by people with the widest range of capabilities. Accessibility is the extent to which an interactive product is accessible by as many people as possible. The primary focus of accessible design is making systems accessible to individuals with disabilities.

Experience-Level Patterns

The user goals that make up the user experience that the application supports, e.g., activity tracking, coaching, and reviewing. Experience-level patterns can also capture the quality of the user experience, e.g., motivational coaching

Simple and Intuitive Use (Princ of Uni Design)

Use of the design is easy to understand, regardless of the user's experience, knowledge, language skills, or current concentration level. 1. Eliminate unnecessary complexity. 2. Be consistent with user expectations and intuition. 3. Accommodate a wide range of literacy and language skills. 4. Arrange information consistent with its importance. 5. Provide effective prompting and feedback during and after task completion.

Gestures: Double-tap

Zooming in/out content, enabling other navigational modes for accessibility purposes, or selecting text.

Accessibility Testing Tools

» WAVE—evaluates the overall level of accessibility for any given website. » Color Oracle— displays your site's colors in a manner similar to how a user with color blindness would see the page. » Image Analyzer— examines website images and tests their compliance with accessibility standards.