Software Testing II

The Testing Plan

-Aligns with the objectives of the project -Aligns with the SDLC chosen -Key issues addressed: --Testing objectives --Environment/Resources --Approaches (what type is used when) --Mechanics of reporting

Reporting

-Classification system for defects -Determine responsibility: --Who reports --Participation in resolution -Human issues in reporting. -Report the defect: not potential resolutions

Test Management Lessons (Kaner, Bach, Petticord)

-Create a service culture -Don't create a control culture -Let the project manager choose the project life cycle -Ask for testability features (this is an upfront budgeting implication)

Equivalence Partitioning Example

-Example of a function which takes a parameter "month." -The valid range for the month is 1 to 12, representing January to December. This valid range is called a partition. -In this example there are two further partitions of invalid ranges. -Test cases are chosen so that each partition would be tested. x < 1 | 1 =< x =< 12 | 12 < x e.g., x = -2, 5, 17

Testing Plan Objectives (A few thoughts)

-Good Enough: According to Bach, this is the most realistic approach, but it is based on a process with measurement. -The test team's primary job is to report, not determine completeness of the product -Base it on the requirements and stakeholders of the system.

Testing Culture

-Intention is to measure a group's inclination -Determine if a discussion or training is needed -Implied context due to propinquity -Overview documents and spread sheet on Blackboard

More Lessons...

-Negotiate the schedule for builds -Adapt processes to the development practices that are actually in use -Don't ask for items unless used -Programmers are like tornadoes -Consider XP practices for automated tests -Enough testing means enough information to make a decision -There is no universal formula for knowing how much testing is enough - nor is there a good rule of thumb for the ratio of testers -Be cautious of measuring a project progress in terms of bug counts

General Techniques

-Specification Based vs. Code Based -Equivalence Classes -Boundary Value Analysis -Decision Table -DD - Paths -Integration Testing Techniques

Integration Testing

-Verify interactions between modules. -Big-Bang: All at once -Decomposition --Top-Down: First call stubs --Bottom-Up: Use of driver modules --Sandwich: Combination of Top-Down and Bottom-Up -Implies the need for well designed components with limited coupling and high cohesion. -Note the benefit of a sequence diagram.

Equivalence Partitioning with Boundary Value Analysis

-We use the same example as before. -Test cases are supplemented with boundary values. x < 1 | 1 =< x =< 12 | 12 < x e.g., x = 0, 1, 2, 11, 12, 13

DD Paths

A DD-path is a set of nodes in a program graph such that one of the following holds: -It consists of a single node with in-degree = 0 (initial node) -It consists of a single node with out-degree = 0 (terminal node) -It consists of a single node with in-degree ≥ 2 or out-degree ≥ 2 (decision/merge points) -It consists of a single node with in-degree = 1 and out-degree = 1 -It is a maximal chain of length ≥ 1.

Studies and Papers Reveal AST Benefits

According to Caper Jones, "without tools, the average tester in the U.S. produces test cases by hand at a rate of about 20 to 300 per month." Poston reported a test case where 24 minutes saved a month of manual labor for a team. (Robert Poston, Automating Specification-Based Software Engineering, IEEE Computer Society Press, 1996)

Equivalence Partitioning

Equivalence partitioning is a software testing technique that divides the input and/or output data of a software unit into partitions of data from which test cases can be derived. The equivalence partitions are usually derived from the requirements specification for input attributes that influence the processing of the test object. Test cases are designed to cover each partition at least once.

Boundary Value Analysis

Equivalence partitioning is not a stand alone method to determine test cases. It is usually supplemented by boundary value analysis. Boundary value analysis focuses on values on the edge of an equivalence partition or at the smallest value on either side of an edge.

Decision Table Testing

Identifies test cases through logical truth table. Example: Benefit Package | Full Time | Days Tenured A | Y | >90 B | Y | =<90 C | N | >90 D | N | =<90 Note: Beware of indeterminate conditions.

TMM Layers (Dustin)

Level | Characteristics and Artifacts 1 | -Initial. -Testing and debugging are interleaved. 2 | -Testing is separated from debugging. -Testing is planned -Testing typically occurs after completion of code. -Test plans based on completed code 3 | -Testing integrated throughout development cycle -Testing begins at requirements phase. -More focus on specification based testing 4 | -Management and Measurement -Software quality attributes receive focus 5 | -Optimization: continuous improvement -Strong infrastructure

What can be partitioned?

Usually it is the input data that is partitioned. However, depending on the software unit to be tested, output data can be partitioned as well. Each partition shall contain a set or range of values, chosen such that all the values can reasonably be expected to be treated by the component in the same way (i.e. they may be considered 'equivalent').