Chapter 2: Theory of Program Testing

What were some drawbacks of Goodenough and Gerhart's theory?

- A criterion is only reliable and valid if its selects the entire input domain in a single test. This is impractical and exhaustive. - Goodness of test should be individual of the program - reliability and validity are not preserved during the debugging process.

What does testing theory put emphasis on?

- detecting defects through execution - designing test cases from different sources: requirements, code, and input/output domains. - selecting a subset of test case from entire input domain - effectiveness of test selection strategies - test oracles used during test -prioritizing execution of the selected test cases - adequacy analysis of test cases.

What are the two categories of faults?

- logic faults - performance faults

what kind of faults are performance faults?

- missing path flow control - inappropriate path selection - inappropriate or missing action

what are two practical methods for evaluating test adequacy?

1. fault seeding 2. Mutation

logic fault

A program fails because of faults in the system, not because of resources.

reliable criterion

A selection criterion is reliable if and only if every test selected by the criterion is successful OR if no selected test are successful - here consistency is key -...so this implies if one complete test fails, all test will fail

valid criterion

A selection criterion is valid if and only if whenever a program is invalid a criterion selects at least 1 test set which is not successful for the program - in this case validity refers to the ability to produce meaningful results - this implies there is a test set that makes the test successful

What is a control flow path ( path)

A sequence of instructions in a program

Construction fault

Failure to satisfy the design. Meaning the design is correct but the construction of the design is incorrect.

Why do faults occur?

Faults occur due to inadequate understanding of all conditions in which a program must deal Also, due to failure to realize that certain combination conditions need special treatments

performance faults

Faults that lead to the program not producing expected results within resource limit

The ideal test

If from a successful execution of the sample of input domain we can conclude that there are no errors, then the sample contributes to the ideal test

Fault Seeding

The process of intentionally adding known defects to those already in the component or system for the purpose of monitoring the rate of detection and removal, and estimating the number of remaining defects.

Uniformly ideal test selection:

When test selection criterion is both uniformly valid and uniformly reliable

A test set is said to be adequate if it

covers all aspects of the actual computation performed by a program and all computations intended by its specification

Test predicate

description of conditions and combinations of conditions relevant to correct operation of the program

Design fault

failure to satisfy understood requirements; meaning requirements were understood but the design is incorrect based on requirements.

Goodenough and Gerhart theory introduced:

ideal test, validity and reliability, test selection criteria, thorough testing, and 5 categories of program errors.

Performance faults are caused by...

improper implementation

Mutation of code

making changes to the code.

An example of a __________ is our failure to test for a zero divisor before executing a division. If we fail to recognize that a divisor can take a zero value, then we will not have the appropriate piece of code to handle the special case.

missing control flow path

what do test theories emphasize?

prioritizing test cases

Wekyur and Ostrand theory introduced:

refined Gerhart's theory into uniformly reliable, valid, ideal test.

what kind of faults are logic faults?

requirement faults - design fault -construction fault

what documents can be used to design a test case?

requirement specifications, source code, and looking at input/output domain.

requirement fault

this fault is due to failure to understand/capture the customers real requirements

Missing control flow path fault

we fail to identify or create a path to handle a condition