Introduction to Software Engineering Test 3

What are the 4 steps of testing?

- Unit testing - Integration testing - High-order testing group (steps 3 and 4) -Validation testing -Systems testing

What are the benefits of smoke testing?

-Integration risk is minimized. -The quality of the end product is improved. -Error diagnosis and correction are simplified. -Progress is easier to assess.

What are the 5 focuses of unit testing?

-Module interface -Local data structures -Boundary conditions -All independent paths -Error-handling paths

What are the roles of the SQA group?

-Prepares an SQA plan for a project. -Participates in the development of the project's software process description. -Reviews software engineering activities to verify compliance with the defined software process. -Audits designated software work products to verify compliance with those defined software process. -Ensures that deviations in software work and work products are documented and handled according to a documented procedure. -Records any noncompliance and reports to senior management.

What is the test specification comprised of?

-Test plan -Test procedure -Test Report

What are the different strategies of integrating for OO testing?

-Thread based testing -Use based testing -Cluster testing

Bottom Up Integration

-drivers are replaced one at a time, "depth first" -worker modules are grouped into builds and integrated.

Top Down Integration

-top module is tested with stubs -stubs are replaced one at a time, "depth first" -as new modules are integrated, some subset of tests is re-run

Validation

A different set of tasks that ensure that the software has been built is traceable to customer requirements; Are we building the right product?

Six-Sigma for Software Engineering

Define Measure Analyze Improve Control

How does effort differ with reviews?

Effort expended with reviews will be slightly increased during the design and coding process, however, the reduced amount of effort and time needed to test and deploy the product is significant enough to say that the reviews are well worth the effort.

Systems testing

Elements mesh and achieves overall system function/performance; The second step of high-order testing and the fourth step of the testing process

Effective Software Process

Establishes the infrastructure that supports any effort at building a high-quality software product.

Design quality

Every element of the design model should be assessed by the software team to ensure that it exhibits high quality and that the design itself conforms to requirements.

Scheduling Decisions

Failing to pay attention to task dependencies when creating the project schedule

For Web Applications

From TITS to TITL: Content -> Interface -> Navigation -> Functional component -> Compatibility -> Performance -> Stress

Quality Factors

Functionality Reliability Usability Efficiency Maintainability Portability

Three Types of Costs in Quality

Prevention costs, Internal failure costs, and External failure costs

Who all is involved in a review meeting?

Producer, review leader, reviewer(s), and recorder

Project Management

Project plan includes explicit techniques quality and change management

Two types of software quality

Quality of design and quality of conformance

Quality control

Series of inspections, reviews, and tests used to ensure conformance of a work product to its specifications

What causes software quality?

Software quality is the result of good project management and solid engineering practice

Code quality

Source code and related work products must conform to local coding standards and exhibit characteristics that will facilitate maintainability.

Elements of SQA

Standards Reviews and Audits Testing Error/defect collection and analysis Change management Education Vendor management Security management Safety Risk Management

Reliability

The amount of time that the software is available for use as indicated by the following sub-attributes: maturity, fault, tolerance, and recoverability.

Informal Review

This is typically done with either a deck check of a software engineering work with a colleague, a casual meeting for the purpose of reviewing a work product, or the review-oriented aspects of pair programming.

Software Engineering MEthods

To build high quality software, you must understand the problem to be solved and be capable of creating a quality design that conforms to the problem requirements Eliminating architectural flaws during design can improve quality

What are the objectives of a Formal Technical Reviews?

To uncover errors in function, logic, or implementation for any representation of the software To verify that the software under review meets its requirements To ensure that the software has been represented according to predefined standards To achieve software that is developed in a uniform manner To make projects more manageable

Quality of conformance

an issue focused primarily on implementation

What is software quality assurance?

an umbrella activity that is applied throughout the software process. it is also called quality management

User satisfaction

compliant product, good quality, and delivery within the budget and schedule

External failure costs

compliant resolution, product return and replacement, help line support, and warranty work

Quality assurance

consists of the auditing and reporting procedures used to provide management with data needed to make proactive decisions

Local data structures

data maintains integrity during all of execution

"Good Enough" Software

delivers high quality functions and features that end-users desire, but at the same time it delivers other more obscure or specialized functions and features that contain known bugs.

Useful Product

delivers the content, functions, and features that the end-user desires

Three important points of software quality

effective software process useful product adds value for the producer and the user

Quality of design

encompasses requirements, specifications, and the design of the system

Types of management decisions

estimation decisions, scheduling decisions, and risk-oriented decisions

All independent paths

execute all paths at least once

Adding Value

high quality software provides benefits for the software organization and the end-user community

Module interface

information flow in and out of unit

Boundary conditions

module operates properly at boundaries

Prevention costs

quality planning, formal technical reviews, test equipment, and training

Regression testing

re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects

Risk-oriented Decisions

reacting to each crisis as it arises rather than building in mechanisms to monitor risks may result in products having reduced quality

Internal failure costs

rework, repair, and failure mode analysis

Error-handling paths

tested and correct

The FTR is actually a class of reviews that includes what?

walkthroughs and inspections

What are reviews?

A meeting conducted by technical people for technical people A technical assessment of a work product during the software engineering process A software quality assurance mechanism A training ground

What are not examples of reviews?

A project summary or progress assessment A meeting intended solely to impart information A mechanism for political or personal reprisal

Software safety

A software quality assurance activity that focuses on the identification and assessment of potential hazards that may affect software negatively and cause an entire system to fail.

Quality control effectiveness

A software team should apply limited resources in a way that has the highest likelihood of achieving a high quality result.

The Review Meeting will result in one of three things:

Acceptance of the product Reject the product due to severe errors Accept the product provisionally

Test Report

Actual test results, problems, and peculiarities are recorded

What does software quality assurance encompass?

An SQA process Specific quality assurance and quality control tasks Effective software engineering practice Control of all software work products and the changes made to them A procedure to ensure compliance with software development standards Measurement and reporting mechanisms

Software Quality

An effective software process applied in a manner that creates a useful product that provides measurable value for those who produce it and those who use it

What is the difference between an error and a defect?

An error is a quality problem found before the software is released to end users, but a defect is a quality problem found only after the software has been released to end-users.

Smoke Testing

An integration testing approach that is commonly used when product software is developed. It is designed as a pacing mechanism for time-critical projects, allowing the software team to asses the project on a frequent basis

What should you do to perform effective testing?

Conduct effective technical reviews. This will lead to many errors being eliminated before testing commences.

Who is the recorder?

The reviewer who records all important issues raised during the review in writing.

Verification

The set of tasks that ensure that software correctly implements a specific function; Are we building the product right?

Smoke Testing Steps

1.) Software components that have been translated into code are integrated into a "build". 2.) A series of tests is designed to expose errors that will keep the build from properly performing its function. 3.) The build is integrated with other builds, and the entire project is smoke tested daily.

What is the best way to do integration testing?

Incremental construction

General Testing Criteria

Interface integrity Functional validity Information content Performance

Interface integrity

Internal and external interfaces are tested as each module is integrated

Estimation Decision

Irrational delivery date estimates cause teams to take short-cuts that can lead to reduced product quality

How does pair programming benefit quality?

It encourages continuous review as a work product is created and provides the benefit of finding many errors immediately.

Mean Time Between Failures (MTBF)

MTBF = MTTF + MTTR (MTTF - Mean time to failure, MTTR - Mean time to repair)

Validation testing

Meets functional, behavior, and performance requirements; The first step of high-order testing and the third step of the testing process

Test procedure

Order of integration and corresponding test at each integration step is described

The Software Quality Dilemma

People in the industry try to get a magic middle ground where a product is good enough to not be immediately rejected, but also not perfect the object so that it doesn't take too long. This results in good enough software.

What are the 4 SQA goals?

Requirements quality Design quality Code quality Quality control effectiveness

For conventional software

TITS: The module (component) TITL: Integration of modules

For OO software

TITS: an OO class that encompasses attributes and operations TITL: The communication and collaboration of said OO class

Information content

Test for errors in local or global data structures

Incremental Construction

Test program in small increments which makes errors easier to isolate and correct; Examples include top-down integration, bottom-up integration, and sandwich testing

Functional validity

Test to uncover functional defects in the software

How does testing start?

Testing begins at the component level and works "outward" toward the integration of the entire computer-based system

Test Plan

Testing divided into phases and builds that address specific functionality and behavioral characteristics

Unit testing

Tests component control structures and is the first step of the testing process.

Requirements quality

The correctness, completeness, and consistency of the requirements model will have a strong influence on the quality of all work products that follow.

Usability

The degree to which the software is easy to use as indicated by the following sub-attributes: understandability, learnability, and operability.

Efficiency

The degree to which the software makes optimal use of system resources as indicated by the following sub-attributes: time behavior, resource behavior.

Functionality

The degree to which the software satisfies stated needs as indicated by the following sub-attributes: suitability, accuracy, interoperability, compliance, and security

Maintainability

The ease with which repair may be made to the software based on analyzability, changeability, stability, and testability.

Portability

The ease with which the software can be transposed from one environment to another based on adaptability, installability, conformance, and replaceability.

How do the different types of cost of quality vary in cost.

The relative costs to find and repair an error or defect increase dramatically as we go from prevention to detection to internal failure to external failure costs.

Who is the producer?

The individual who has developed the work product. Also informs the project leader that the work product is finished and requires review.

Who is/are the reviewer(s)?

The people who are expected to spend between one and two hours reviewing the product, making notes, and otherwise becoming familiar with the work.

Who is the review leader?

The person who evaluates the product for readiness, generates copies of product materials, and distributes them to two or three reviewers for advance preparation.

Software availability

The probability that a program is operating according to requirements at a given point in time

Software testing

The process of exercising a program with the specific intent of finding errors prior to the delivery to the end user

Defect Amplification Model

Used to illustrate the generation and detection of error during the design and code generation actions of a software process.

Integration testing

Verification, program construction, and the second step of the testing process

Performance

Verify specified performance bounds are tested

How do we test?

We begin by 'testing-in-the-small' (TITS) and move toward 'testing-in-the-large' (TITL)

Availability

[MTTF/(MTTF+MTTR)]X100%

Quality (dictionary definition)

a characteristic or attribute of something