Chapter 7: Integration testing

Interface between two modules

An interface allows one module to access the service provided by the other. It implements a mechanism for passing control and data between modules. Three types of interfacing modules: - Procedure call interface: a procedure in one module calls a procedure in another module. The caller passes on control to the called module. The caller can pass data to the called procedure, and the called procedure can return data when returning control. - Shared memory interface: a block of memory is shared between two modules, so that data are written to the memory block by one module and read by another. - Message passing interface: one module prepares a message by initializing the fields of a data structure and sending the message to another module.

Objective of integration testing

To assemble a reasonably stable system in a laboratory environment such that the integrated system can withstand the rigor of a full-blown system testing in the actual environment of the system.

Integration testing

Integration testing is a systematic technique for assembling a software system while conducting tests to uncover errors associated with interfacing. A working version of the system is built by putting the modules together incrementally and ensuring that the additional modules work as expected without disturbing the functionalities of the modules already put together. Integration system is complete when the system is fully integrated together, all test cases have been executed, all severe and moderate defects have been fixed and the system is retested. Integration testing is performed by the system integration group, also known as a build engineering group. The developers who built the modules need to be involved since details of the modules need to be known. In complex systems, system architects should also be involved.

The idea behind check-in request mechanism in the incremental integration approach

- All the files requiring an update must be identified and known to other team members - The new code must have been reviewed prior to its integration - The new code must have been unit tested - The scope of the check-in is identified

Advantages of integration testing

- Defects are detected early - It is easier to fix defects detected early - We get earlier feedback - Scheduling of defect fixes is flexible and can overlap with development

Three reasons why integration testing is important

- Different modules are generally created by groups of different developers - Unit testing of individual modules is carried out in a controlled environment by using test drivers and stubs - Some modules are more error prone than other modules, because of their inherent complexity

Comparison of top-down and bottom-up approaches

- Faults in system design decisions are detected early in top-down and towards the end in bottom-up - System-level functions are observed early in top-down and towards the end in bottom-up - It becomes increasingly difficult to design stub behavior and test input in top-down, and in bottom-up one designs the behavior of a test driver by simplifying the actual module's behavior - In top-down, test cases are reused in regression test which saves time and money. In bottom-up, test cases cannot be used.

5 integration techniques

- Incremental: in each test cycle, a few more modules are integrated with an existing and tested build to generate a larger build. Every new core is intended to be self-contained (containing all necessary code for a certain set of functions) and stable (the subsystem should not have any anomalies). The developer fills out a check-in request form before a new/fixed module is integrated, which is reviewed by the build group. - Top down: starting with the top-level module and replace one of its stubs at a time, to discover any interface defects between them and to ensure that it does not break anything. - Bottom up: integration starts with the lowest level modules (modules that do not invoke other modules). The drivers need to be constructed and then replaced with the actual modules when going up in the hierarchy, which continues until all modules have been integrated. - Sandwich: a mix of top-down and bottom-up. A hierarchical system is view as consisting of three layers: the bottom layer with all the modules that are often invoked (where bottom-up approach is applied to integrate the modules), the top layer with the modules with major design decisions (where top-down approach is applied) and a middle layer with the rest (where big bang approach is applied) - Big bang: first all modules are individually tested, then they are put together to construct the entire system which is tested as a whole. This can work for small systems.

3 levels of granularity for integration testing

- Intrasystem testing: low-level integration testing with the objective of combining the modules together to build a cohesive system - Intersystem testing: high-level testing phase which requires interfacing independently tested systems - Pairwise testing: an intermediate level of integration testing, where only two interconnected systems in an overall system are tested at a time.

Advantages of the bottom-up approach

- Low-level modules are invoked by other modules, which makes it useful to test them first. Stubs only provide a limited test capability of the interfaces.

Parameters determining the number of integration test cycles and total integration time in incremental integration approach

- Number of modules - Relative complexity of the modules - Relative complexity of the interfaces between the modules - Number of modules needed to be clustered together in every cycle - Wether the modules have been adequately tested before - Turnaround time for test-debug-fix cycle

Disadvantages of the bottom-up approach

- System-level functions cannot be observed from a partly integrated system - Major design decisions are embodied in the top-level modules, so discovery of major flaws in system design may not be possible until the top-level modules have been integrated

Advantages of the top-down integration approach

- Test engineers continually observe system-level functions as the integration process continues, and early observation of system functions gives them better confidence - Isolation of interface errors become easier - Test cases designed to test the integration of one module can be reused during the regression test when other modules are integrated - Since test inputs are applied to the top-level module, they normally correspond to system functions

Reasons not to use big bang integration approach for large systems

- There may be many interface defects, and it is difficult to determine whether or not the cause of a failure is due to interface errors - It is not cost effective to be optimistic since many interface errors probably exist

Limitations of the top-down approach

- Until a certain set of modules has been integrated, it may not be possible to observe meaningful system functions - Test case selection and stub design become increasingly difficult far away from the top-level module

Empirical study of relative efficacy between different integration approaches

- Top-down is most effective in defect correction - Top-down and big-bang produce the most reliable systems - Bottom-up is the least effective in defect correction and it produces the least reliable systems - Sandwich is moderately reliable

Issues addressed while planning a test strategy during incremental integration approach

- What test cases need to be selected from the integration test plan in order to test the changes? - What existing test cases can be reused to test the modified system? - How should the scope of a partial regression test be determined? - What are the estimated time, resource demand and cost to test the new build?

Interface errors

Errors that are associated with structures existing outside the local environment of a module but which the module uses. Examples of interface errors: - Inadequate functionality: implicit assumptions in one part of a system that another part of the system would perform - Location of functionality: disagreement on/misunderstanding about the location fo functionality - Changes in functionality: changing one module without adjusting related modules Interface errors cannot be detected during unit testing. It is also difficult to detect them during system testing, since they tend to be buried in system internals.