Chapter 9 - Testing
testNG
-based on annotations -flexible runtime config -introduces the notion of "test groups" to clearly separate static testing (implementing the methods) from dynamic (which tests are run)
prevention, recovery, detection
3 goals of the types of testing
system, validation, integration, unit
4 types of testing
big bang, bottom up, top down, sandwich, continuous
5 types of horizontal integration testing strategies:
unit
__ testing: -individual component (class/subsystem) -carried out by devs -junit -static/dynamic techniques
sandwich
___ approach to horizontal integration: •Combines top-down strategy with bottom-up strategy •Make use of the best of both •The system is viewed as having three layers -A target layer in the middle -A layer above the target -A layer below the target •Testing converges at the target layer.
big bang
___ approach to horizontal integration: •Assume all components are first tested individually then tested together •No additional test stubs or drivers are needed •Simple but expensive - Impossible to distinguish failures in the interface from failures within components. -Hard to pinpoint a specific component or a combination of components
top down
___ approach to horizontal integration: •Test the top layer or the controlling subsystem first •Then combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems •Do this until all subsystems are incorporated into the test •Stubs are needed to do the testing.
bottom up
___ approach to horizontal integration: •The subsystems in the lowest layer (bottom layer) of the call hierarchy are tested individually •Then the next subsystems are tested that call the previously tested subsystems •This is repeated until all subsystems are included •Drivers are needed to simulate the components of higher layers that have not yet been integrated
black box
___ dynamic unit testing: -design of software tests relies on description -inputs, functionality, outputs -equivalence partitioning, boundary value analysis, random testing, cause-effect graphing -focus: I/O behavior -for any given input we can predict the output
white box
___ dynamic unit testing: -design of software tests relies on source code -analyze module algorithm in detailed design -path testing, mutation testing
vertical
___ integration testing: •Components are integrated according to functions
horizontal
___ integration: -Components are integrated according to layers . Not hard to manage, e.g., verify interfaces between teams a.Big bang b.Bottom up c.Top down d.Sandwich e.Continuous
equivalence
___ testing: black box technique. partition possible inputs into classes and a test case selected for each class
boundary
___ testing: black box technique. testing special case of equivalence classes (edges)
state-based
___ testing: select inputs for given state of system and compare results with expected
path testing
____ testing: white box technique. identifies faults in implementation of component. exercise all possible paths through code. statement, edge, condition, path coverage
stub
a test ___ simulates the components that are called by the tested component
driver
a test ___ simulates the part of the system that invokes operation on the tested component
•Does not test the individual subsystems and their interfaces thoroughly before integration
con of sandwich testing
-Tests the most important subsystem (user interface) last -Drivers needed
cons of bottom up testing
•Writing stubs is difficult: Stubs must allow all possible conditions to be tested. •Large number of stubs may be required, especially if the lowest level of the system contains many methods. •Some interfaces are not tested separately.
cons of top down testing
•Top and Bottom Layer Tests can be done in parallel
pro of sandwich testing
-No stubs needed -Useful for integration testing of the following systems •Object-oriented systems •Real-time systems •Systems with strict performance requirements.
pros of bottomup testing
•Test cases can be defined in terms of the functionality of the system (functional requirements) •No drivers needed
pros of top down testing
black-box and white-box (functional and structure)
two types of dynamic unit testing:
horizontal and vertical
two types of integration testing
symbolic execution, program proving, anomaly analysis
types of static unit testing: ____- works for small programs ____- difficult to apply, not guarantee correctness ____-inexecutable code, uninitialized vars, unused vars
-unit tests only test unit in isolation -many failures result from faults in interaction of subsystems -Often many Off-the-shelf components are used that cannot be unit tested
why do we do integration testing?