Chapter 9 - Testing

testNG

-based on annotations -flexible runtime config -introduces the notion of "test groups" to clearly separate static testing (implementing the methods) from dynamic (which tests are run)

prevention, recovery, detection

3 goals of the types of testing

system, validation, integration, unit

4 types of testing

big bang, bottom up, top down, sandwich, continuous

5 types of horizontal integration testing strategies:

unit

__ testing: -individual component (class/subsystem) -carried out by devs -junit -static/dynamic techniques

sandwich

___ approach to horizontal integration: •Combines top-down strategy with bottom-up strategy •Make use of the best of both •The system is viewed as having three layers -A target layer in the middle -A layer above the target -A layer below the target •Testing converges at the target layer.

big bang

___ approach to horizontal integration: •Assume all components are first tested individually then tested together •No additional test stubs or drivers are needed •Simple but expensive - Impossible to distinguish failures in the interface from failures within components. -Hard to pinpoint a specific component or a combination of components

top down

___ approach to horizontal integration: •Test the top layer or the controlling subsystem first •Then combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems •Do this until all subsystems are incorporated into the test •Stubs are needed to do the testing.

bottom up

___ approach to horizontal integration: •The subsystems in the lowest layer (bottom layer) of the call hierarchy are tested individually •Then the next subsystems are tested that call the previously tested subsystems •This is repeated until all subsystems are included •Drivers are needed to simulate the components of higher layers that have not yet been integrated

black box

___ dynamic unit testing: -design of software tests relies on description -inputs, functionality, outputs -equivalence partitioning, boundary value analysis, random testing, cause-effect graphing -focus: I/O behavior -for any given input we can predict the output

white box

___ dynamic unit testing: -design of software tests relies on source code -analyze module algorithm in detailed design -path testing, mutation testing

vertical

___ integration testing: •Components are integrated according to functions

horizontal

___ integration: -Components are integrated according to layers . Not hard to manage, e.g., verify interfaces between teams a.Big bang b.Bottom up c.Top down d.Sandwich e.Continuous

equivalence

___ testing: black box technique. partition possible inputs into classes and a test case selected for each class

boundary

___ testing: black box technique. testing special case of equivalence classes (edges)

state-based

___ testing: select inputs for given state of system and compare results with expected

path testing

____ testing: white box technique. identifies faults in implementation of component. exercise all possible paths through code. statement, edge, condition, path coverage

stub

a test ___ simulates the components that are called by the tested component

driver

a test ___ simulates the part of the system that invokes operation on the tested component

•Does not test the individual subsystems and their interfaces thoroughly before integration

con of sandwich testing

-Tests the most important subsystem (user interface) last -Drivers needed

cons of bottom up testing

•Writing stubs is difficult: Stubs must allow all possible conditions to be tested. •Large number of stubs may be required, especially if the lowest level of the system contains many methods. •Some interfaces are not tested separately.

cons of top down testing

•Top and Bottom Layer Tests can be done in parallel

pro of sandwich testing

-No stubs needed -Useful for integration testing of the following systems •Object-oriented systems •Real-time systems •Systems with strict performance requirements.

pros of bottomup testing

•Test cases can be defined in terms of the functionality of the system (functional requirements) •No drivers needed

pros of top down testing

black-box and white-box (functional and structure)

two types of dynamic unit testing:

horizontal and vertical

two types of integration testing

symbolic execution, program proving, anomaly analysis

types of static unit testing: ____- works for small programs ____- difficult to apply, not guarantee correctness ____-inexecutable code, uninitialized vars, unused vars

-unit tests only test unit in isolation -many failures result from faults in interaction of subsystems -Often many Off-the-shelf components are used that cannot be unit tested

why do we do integration testing?