Quality Assurance Mid-Term

User Story Syntax

"As a <role>,I want <goal/desire>[so that <benefit>]"

Software Quality Assurance

- A planned and systematic pattern of all actions necessary to provide confidence that an item or product meets established technical requirements.

V-Model

- Acceptance Test Plan - System Test Plan - Integration Testing Plan - Unit Testing Plan - Coding

Test Case Characteristics

- Accurate: Exacts the purpose. - Economical: No unnecessary steps or words. - Traceable: Capable of being traced to requirements. - Reusable: Can be reused if necessary.

Characteristics of a Good Requirement

- Cohesive - Complete - Consistent - Non-Conjugated - Traceable - Current - Unambiguous - Verifiable

Review Objectives

- Identify analysis and design errors - Identify new risks - Locate deviations from templates - Approve the analysis or design of a product - Provide an informal meeting place for exchange of professional knowledge about methods, tools and techniques.

Quality Triangle (Managing the Constraints)

- Time constraint refers to the amount of time available to complete a project. - Cost constraint refers to the budgeted amount available for the project. - Scope constraint refers to what must be done to produce the project's end result. - Competing constraints - Increased scope means increased time/cost - Tight time constraint could mean increased costs and reduced scope - Tight budget could mean increased time and reduced scope.

Agile Manifesto

- Uncovering better ways of developing software by doing it and helping others do it. - Individuals and interactions over processes and tools - Working software over comprehensive documentation - Customer collaboration over contract negotiation - Responding to change over following a plan

Prototyping Advantages

- Users are actively involved in the development. - Users get a better understanding of the system being developed. - Errors can be detected much earlier. - Quicker user feedback

Waterfall Disadvantages

- Very difficult to go back and change something once in testing stage - No working software is produced until late during the life cycle. - High amounts of risk and uncertainty. - Not good for complex object-oriented projects - Not suitable for projects where requirements can change.

What could be an example of a review?

-Static Testing -Unit Testing

Software Quality

System meets specified requirements of customer or user

Graphical User Interface (GUI) Testing

Testing a product's graphical user interface to ensure it meets its specifications.

User Story

An Agile requirement expressed from the user's point of view, and describes a unit of desired functionality.

Functional Requirement Type

Capabilities, behavior, and information that the solution will need.

System Testing

Verifies the system meets its requirements.

Prototyping

Developing prototypes early in the development process to get early feedback and analysis

Design Review

Documented analysis of a design to determine its capability to meet its requirements.

Usability

Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

Function coverage

Has each function (or subroutine) been called?

Path coverage

Has each path in the program, including all loop paths taken zero, once, and multiple times, been executed? Calculation - Find out all paths from start to end

Architectural Requirement Type

Identifies the necessary systems structure and behavior

Peer Reviews

Walkthroughs and Inspections

Forward Traceability

Mapping of requirements to test cases and ensures proper direction of product

Reverse or Backward Traceability

Mapping of test cases to requirement. Check that the end product has met the requirements or not

User (Stakeholder) Requirement Type

Mid-level statements of the needs of a particular stakeholder or group

Software Development Life Cycle (SDLC)

Process for planning, creating, testing, and deploying an information system (or product).

Waterfall Model

Progress is seen as flowing steadily downwards (like a waterfall) through phases

Cost of Quality

Quantify the total cost of quality-related efforts and deficiencies.

Requirement Types

- Architectural - Business - User (Stakeholder) - Functional - Non-Functional

Software Quality Assurance Objectives

- Assuring a level of confidence that the software will meet functional and technical requirements. - Assuring an level of confidence that the software will meet managerial scheduling and budgetary requirements.

Agile Testing Matrix Quadrant's

- Automated & Manual - Automated - Tools - Manual

Requirements Hierarchy Chart

- Business Level - Stakeholder Level - Product Level

Requirements Hierarchy Triangle

- Business Requirements - User Requirements - System Requirements

User Story Three C's Formula

- Card: a physical token giving tangible and durable form to what would only be an abstraction - Conversation: taking place at different time and places during a project between the various stakeholders concerned by the given feature (customers, users, developers, testers, etc.), which is largely verbal but most often supplemented by documentation: - Confirmation: ensures that the objectives the conversation revolved around have been reached finally.

Agile Advantages

- Customer satisfaction by continuous delivery of software. (weeks rather than months) - People and interactions are emphasized rather than process and tools. - Customers, developers and testers constantly interact with each other. - Face-to-face conversation is the best form of communication.

Agile Disadvantages

- Difficult to assess the effort required at the beginning of the software development life cycle. - Lack of emphasis on designing and documentation. - Project can get off track if the customer representative is not clear what final outcome that they want.

Black-Box Test Design Techniques

- Equivalence Partitioning - Boundary Value Analysis

Testing

- Evaluating a product by learning about it through exploration and experimentation - Provide stakeholders with information about the quality of the product under test.

Objectives of Testing

- Finding defects - Prevent defects - Meet business and user requirements. - Gain the confidence of the customers by providing them a quality product.

Agile Development Process

- Gather & Write Requirements - Create User Stories w/ Acceptance Criteria - Write Acceptance Tests - Code - Write Unit Tests - Execute Tests and Report Results - Report a Defect - Fix Defect & Rerun Tests

Test Automation Approaches

- Graphical user interface testing - API driven testing

Challenges of Software Quality Assurance

- High complexity - Very large number of users - Invisibility of the product - Limited opportunities to detect defects ("bugs")

Characteristics of a Good User Story

- INVEST Criteria - Independent: not dependent on another user story - Negotiable: can always be rewritten - Valuable: deliver value to user - Estimable: size of user story - Small: not be big - Testable

Prototyping Disadvantages

- Leads to repairing the system - Scope of the system may expand beyond original plans.

Waterfall Features

- Phase cannot begin until the previous phase is completed. - At the end of every phase, there is a gate where a decision is made to allow the project to move forward or not. - Changes are controlled (Change Control Board has to approve them) - Product is only finished at the end of the last phase. - Once the project is done, the product enters a maintenance phase.

Dynamic Testing

- Program is run by actually executing test cases. - May begin before the program is 100% complete to test sections of code. - involves validation

Why Automate?

- Saves time and money - Improves testing accuracy compared to testing directed by humans - Increases test coverage because multiple testing tools can be deployed at once allowing for parallel testing of different test scenarios - Finding bugs and errors more quickly

Waterfall Advantages

- Simple and easy to understand and use. - Easy to manage and each phase has specific deliverables and a review process. - Phases are completed one at a time. - Phases do not overlap. - Works well for smaller projects

Test Case Elements

- Test Case ID - Test Case Description - Pre-Conditions - Test Step(s) - Test Data - Expected Result - Actual Result - Pass/Fail Status - Comments/Notes

Static Testing

- Testing where the actual program is not used. - involves verification

Agile Requirements Hierarchy

- Visioning - Brainstorm - Breakdown - Deep Dive

Quality Triangle: Waterfall vs. Agile

- Waterfall: scope is fixed, cost and schedule are variables - Agile: scope is a variable, cost and schedule are fixed

Agile Manifesto Principles

- Welcome changing requirements, even late - Deliver working software early and frequently - Business people and developers work together daily - Build projects around motivated individuals - Convey info via face-to-face conversation

Test Case Good Practices

- Write test cases in such a way that you test only one thing at a time, i.e., atomic. - Ensure that all positive scenarios and negative scenarios are covered. - Language:•Write in simple and easy to understand language. - Use active voice: Do this, do that. - Use exact and consistent names (of forms, fields, etc.).

Manual vs Automated Testing

- execution time: automation much quicker - amount of people: manual has much more people - Infrastructure: even - Turnaround Time: manual much quicker - Training: manual much quicker

Quality Triangle

- model of the constraints of project management - Illustrates that project management success is measured by the project team's ability to manage the project, so that the expected results are produced while managing time and cost.

What Not To Automate

- test cases that are newly designed and not executed manually at least once - test cases where requirements are changing frequently - test cases that are executed on ad-hoc basis

Acceptance Tests

- the functional testing of a user story by the agile team. - black-box tests - State an intent, not a solution

Test Automation Waterfall vs Agile

- traditional (find bugs) - agile (prevent bugs)

Non-Functional Requirement Type

Conditions under which the solution must remain effective, qualities that the solution must have, or constraints within which it must operate

Code Coverage

Describes the degree to which the source code of a program is tested by a particular test suite

Validation

Did we build the right software product?

Verification

Did we build the software product right?

Equivalence Partitioning

Divide (i.e. to partition) a set of test conditions into groups or sets that can be considered the same (i.e. the system should handle them equivalently).

Usability Lab

Environment where users are studied interacting with a system for the sake of evaluating the system's usability.

Boehm's First Law

Errors are more frequent during requirements and design stage and are more expensive the later they are removed.

User Acceptance Testing

Evaluate the system's compliance with the business requirements and assess whether it is acceptable for delivery.

Usability Testing

Evaluating a product or service by testing it with representative users.

Exploratory Testing

Exploring, finding out about the software, what it does, what it doesn't do, what works and what doesn't work

Regression Testing

Finding defects after a code change has occurred.

Acceptance Criteria Syntax

Given-When-Then formula

Branch coverage

Has each branch of each control structure (such as in if and case statements) been executed? For example, given an if statement, have both the true and false branches been executed? Calculation - Find out the minimum number of paths which will ensure covering of all the edges.

Statement coverage

Has each statement in the program been executed? Calculation - Find out the shortest number of paths in which all the nodes will be covered.

Business Requirement Type

High-level statements of the goals, objectives, or needs of an organization

Black-Box Testing

Treats the software as a "black box", examining functionality without any knowledge of internal implementation.

Test Automation

Use of special software to control the execution of tests and the comparison of actual outcomes with predicted outcomes.

Integration Testing

Verifies the interfaces between components against a software design.

Unit Testing

Verify the functionality of a specific section of code

Inspections

Verifying products by manually examining the developing product, a piece at a time, by small groups of peers to ensure that it meets product specifications and requirements.

Agile Testing Manifesto

we value: - testing throughout over at the end - preventing bugs over finding bugs - testing understanding over checking functionality - building the best system over breaking the system - team responsibilityover tester responsibility

Software Requirements Specification (SRS)

description of a software system to be developed.

Requirements

A condition needed by a user to solve a problem or achieve an objective.

Requirements Traceability Matrix

A document, usually in the form of a table, that correlates any two baselined documents that require a many-to-many relationship

Agile

Small teams delivering small increments of working software with great frequency while working in collaboration with customers and adapting to changing requirements.

Walkthroughs

Software peer review where a designer or programmer leads members of the development team through a software product, and the participants ask questions

White-Box Testing

Tests internal structures of a program as opposed to the functionality exposed to the end-user.

Agile Testing Matrix

The Quadrants can be used to: - As a communication tool to the project team and management - Explain testing in a common language - Emphasize whole-team responsibility/participation - Focus on collaboration

Requirements Volatility

additions, deletions and modifications of requirements

Test Automation Framework

an integrated system that sets the rules of automation of a specific product.

Requirements Coverage

the percent of requirements covered by formal test cases.

Control flow diagram (CFD)

diagram to describe the control flow of a (business) process or (software) program. - Nodes represent processing tasks while edges represent control flow between the nodes. 1. - The "Start" and "End" symbols in a control flow diagram should not be counted as separate nodes, but should be included when counting edges. 2 .If-Then-Else statements do not and should not have an "Endif" statement, as the "Else" effectively ends this statement and an "endif" is unnecessary and redundant. 3.Do not embed an implied an "endif" to accompany every if statement in a program, as this will increase the count of nodes and edges accordingly.

Inverted Test Automation Pyramid

occurs when too much emphasis is placed on automated GUI & manual tests.

Requirements Engineering

refers to the process of defining, documenting and maintaining requirements.

Requirements Traceability Benefits

saves time, money, and effort.

Test Case

set of conditions under which a test will determine whether an application, software system or one of its features is working as it was originally established to do.

Cyclomatic complexity

software measurement used to indicate the complexity of a program. Calculation = E - N + 2

Boundary value analysis (BVA)

testing at the boundaries between partitions. - valid boundaries (in the valid partitions) - invalid boundaries (in the invalid partitions).

Requirements Traceability

the ability to describe and follow the life of a requirement, in both forwards and backwards direction

Acceptance Criteria

the conditions that a software product must satisfy to be accepted by a user, customer, or in the case of system level functionality, the consuming system.