Chapter 2: Theory of Program Testing
What were some drawbacks of Goodenough and Gerhart's theory?
- A criterion is only reliable and valid if its selects the entire input domain in a single test. This is impractical and exhaustive. - Goodness of test should be individual of the program - reliability and validity are not preserved during the debugging process.
What does testing theory put emphasis on?
- detecting defects through execution - designing test cases from different sources: requirements, code, and input/output domains. - selecting a subset of test case from entire input domain - effectiveness of test selection strategies - test oracles used during test -prioritizing execution of the selected test cases - adequacy analysis of test cases.
What are the two categories of faults?
- logic faults - performance faults
what kind of faults are performance faults?
- missing path flow control - inappropriate path selection - inappropriate or missing action
what are two practical methods for evaluating test adequacy?
1. fault seeding 2. Mutation
logic fault
A program fails because of faults in the system, not because of resources.
reliable criterion
A selection criterion is reliable if and only if every test selected by the criterion is successful OR if no selected test are successful - here consistency is key -...so this implies if one complete test fails, all test will fail
valid criterion
A selection criterion is valid if and only if whenever a program is invalid a criterion selects at least 1 test set which is not successful for the program - in this case validity refers to the ability to produce meaningful results - this implies there is a test set that makes the test successful
What is a control flow path ( path)
A sequence of instructions in a program
Construction fault
Failure to satisfy the design. Meaning the design is correct but the construction of the design is incorrect.
Why do faults occur?
Faults occur due to inadequate understanding of all conditions in which a program must deal Also, due to failure to realize that certain combination conditions need special treatments
performance faults
Faults that lead to the program not producing expected results within resource limit
The ideal test
If from a successful execution of the sample of input domain we can conclude that there are no errors, then the sample contributes to the ideal test
Fault Seeding
The process of intentionally adding known defects to those already in the component or system for the purpose of monitoring the rate of detection and removal, and estimating the number of remaining defects.
Uniformly ideal test selection:
When test selection criterion is both uniformly valid and uniformly reliable
A test set is said to be adequate if it
covers all aspects of the actual computation performed by a program and all computations intended by its specification
Test predicate
description of conditions and combinations of conditions relevant to correct operation of the program
Design fault
failure to satisfy understood requirements; meaning requirements were understood but the design is incorrect based on requirements.
Goodenough and Gerhart theory introduced:
ideal test, validity and reliability, test selection criteria, thorough testing, and 5 categories of program errors.
Performance faults are caused by...
improper implementation
Mutation of code
making changes to the code.
An example of a __________ is our failure to test for a zero divisor before executing a division. If we fail to recognize that a divisor can take a zero value, then we will not have the appropriate piece of code to handle the special case.
missing control flow path
what do test theories emphasize?
prioritizing test cases
Wekyur and Ostrand theory introduced:
refined Gerhart's theory into uniformly reliable, valid, ideal test.
what kind of faults are logic faults?
requirement faults - design fault -construction fault
what documents can be used to design a test case?
requirement specifications, source code, and looking at input/output domain.
requirement fault
this fault is due to failure to understand/capture the customers real requirements
Missing control flow path fault
we fail to identify or create a path to handle a condition
