Software Testing Assurance

Testing here focuses on the ___ of the system; the ___ of the program is not emphasized. Consequently, mostly functional testing is performed at these levels.

external behavior, internal logic

There are ___ different types of test strategies

five

A history of actual test results, problems, or peculiarities is recorded in the ___ of the Test Specification. Information contained in this section can be vital during software maintenance.

fourth section

Test Specification outline that may be used as a ___.

framework

Each of these phases and sub-phases (denoted in parentheses) delineates a broad ___ within the software and can generally be related to a ___ of the program structure.

functional category, specific domain

Both the plan and the procedure are designed to ensure that all ___ are satisfied, all ___ are achieved, ___ is correct and ____ , and other requirements are met (e.g., transportability, compatibility, error recovery, maintainability).

functional requirements, performance requirements, documentation, documentation

The Test plan section describes the overall strategy for ___.

integration

It is important to note, however, that an ___, contained in a Test plan, and testing details, described in a Test procedure, are essential ingredients and must appear.

integration strategy

Regression testing ___.

is the activity that helps to ensure that changes (due to testing or other reasons) do not introduce unintended behavior or additional errors

Proper application of ___ all lead to quality that is confirmed during testing.

methods and tools, effective formal technical reviews, and solid management and measurement

Strict top-down testing is difficult to implement because

of the requirement that program stubs, simulating lower levels of the system, must be produced.

A brief description of ___ concentrates on characteristics that might require special effort. Finally, the test environment and resources are described.

overhead software (stubs-and drivers)

Testing is divided into ___ and ___ that address specific ___ and ___ characteristics of the software.

phases and builds, functional and behavioral

___ are created to correspond to each phase.

program builds (groups of modules)

Appropriate ___ and ___ are presented in the final two sections.

references and appendices

Correction of these faults might involve the ___ and ___ of low-level modules in the system.

rewriting and consequent re-testing

The basic levels are ___. These different levels of testing attempt to detect different types of faults.

unit testing, integration testing and system and acceptance testing

Validation can be defined in many ways, but a simple (albeit harsh) definition is that ___.

validation succeeds when the software functions in a manner that can be reasonably expected by the customer

Formal technical reviews some times called a ___ or ___. This is most effective filter from a quality assurance standpoint.

walkthrough or an inspection

Designing ___ requires thorough knowledge of the internal structure of software, and therefore white-box testing is also called the structural testing.

white-box test cases

The following criteria and corresponding tests are applied for all test phases:

Interface integrity. Internal and external interfaces are tested as each module (or cluster) is incorporated into the structure. Functional validity. Tests designed to uncover functional errors are conducted. Information content. Tests designed to uncover errors associated with local or global data structures are conducted. Performance. Tests designed to verify performance bounds established during software design are conducted.

___ can be as effective as testing in uncovering errors. For this reason, reviews can reduce the amount of testing effort that is required to produce high-quality software. Software reviews are a "filter" for the software engineering process. That is, reviews are applied at various points during software development and serve to uncover errors and defects that can then removed. Softer reviews "purify" the software engineering activities that we have called analysis, design, and coding. Many different types of reviews can be conducted. An informal meeting around the coffee machine is a form of review, if technical problems are discussed. A formal presentation of software design to audience of customers, management, and technical staff is, also a form of review.

Formal technical reviews

Acceptance testing

In acceptance testing, the entire software system is tested. The reference document for this process is the requirements document, and the goal is to see if the software meets its requirements. This is essentially a validation exercise, and in many situations it is the only validation activity. Acceptance testing is sometimes performed with realistic data of the client to demonstrate the software working satisfactorily.

____ is based upon the concepts like objects and attributes, classes and numbers, wholes and parts.

OOA-object oriented analysis

The main disadvantage of top-down testing is that test out put may be

difficult to observe. In many systems, the higher levels of that system do not generate output but, to test these levels, they must be forced to do so. The tester must create an artificial environment to generate the test results.

Miller relates software testing to quality assurance by stating that

"the underlying motivation of program testing is to affirm software quality with methods that can be economically and effectively applied to both large-scale and small-scale systems."

This type of testing has two functions:

(1) It tests the failure behavior of the system. (2) It stresses the system and may cause defects to come to light, which would not normally manifest themselves.

Back-to-back testing is only usually possible in the following situations:

(1) When a system prototype is available. (2) When reliable systems are developed using N-version programming. (3) When different versions of a system have been developed for different types of computers.

Using white-box methods, the software engineer can derive test cases that

1. Guarantee that all independent paths within a module have been exercised at least once, 2. Exercise all logical decisions on their true and false sides, 3. Execute all loops at their boundaries and within their operational bounds, and 4. Exercise internal data structures to ensure their validity.

After each validation test case has been conducted, one of the two possible conditions exists:

1. The function or performance characteristics conform to specification and are accepted, or 2. A deviation from specification is uncovered and a deficiency list is created.

In order to build an analysis model five basic principles were applied:

1. The information domain is modeled; 2. Function is described; 3. Behavior is represented; 4. Data, functional and behavioral models are partitioned to expose greater details; and 5. Early models represent the essence of the problem while later models provide implementation details

The changes that will affect software engineering will be influenced from four simultaneous sources:

1. The people who do not work, 2. The process that they apply, 3. The nature of information, 4. The under laying computing technology.

System testing

A strategy for software testing may also be viewed as in the context of the spiral (Figure 9.4) unit testing begins at the vertex of the spiral and concentrated on each unit by moving outward along the spiral to integration testing, where the focus is on design and the construction of software architecture. Taking another run outward on the spiral, we encounter validation testing, where requirements established as part of software requirements analysis are validated against the software that has been constructed. Finally, we arrive at system testing, where the software and other system elements are tested as a whole. To test computer software, we spiral out along streamlines that broaden the scope of testing with each turn. System testing verifies that all elements mesh properly and that overall system function/performance is achieved.

The following are the list of testing principles.

All tests should be traceable to customer requirements. As we have seen, the objective of software testing is to uncover errors. It follows that the most severe defects (from the customer‟s point of view) are those that cause the program to fail to meet its requirements.

___ are unlikely to be discovered until much of the system has been tested.

Architectural faults

Back-to-back testing

Back-to-back testing may be used when more than one version of a system is available for testing. The same tests are presented to both versions of the system and the test results compared.

Black-box testing (Functional testing)

Black box testing alludes to tests that are conducted at the software interface. Although they are designed to uncover errors, black box tests are used to demonstrate that software functions are operational, that input is properly accepted and output is correctly produced, and that the integrity of external information (e.g., a database) is maintained. A black-box test examines some fundamental aspect of a system with little regard for the internal logical structure of the software. In the black-box approach, test cases are designed using only the functional specification of the software, i.e. without any knowledge of the internal structure of the software. For this reason black-box testing is also known as functional testing.

Bottom-Up Testing

Bottom - up testing is the converse of top down testing. It involves testing the modules at the lower levels in the hierarchy, and then working up the hierarchy of modules until the final module is tested.

___ is appropriate for object-oriented systems in that individual objects may be tested using their own test drivers they are then integrated and the object collection is tested. The testing of these collections should focus on object interactions.

Bottom-up testing

___ of critical, low-level system components is almost always necessary.

Bottom-up testing

Regression Testing

Each time a new module is added as part of integration testing, the software changes. New data flow paths are established, new I/O may occur, and new control logic is invoked. These changes may cause problems with function that previously worked flawlessly. In the context of an integration test strategy, regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects.

___ is incorporated into software throughout the process of software engineering.

Quality

summarizes the specific functional, performance, and internal design characteristics that are to be tested.

Scope of testing

Reasonable expectations are defined in the ___ - a document that describes all user-visible attributes of the software. The Specification contains a section called ___.

Software Requirements Specification, Validation criteria

___ is one element of a broader topic that is often referred to as verification and validation (V&V).

Software testing

Stress testing

Some classes of system are designed to handle specified load. For example, a transaction processing system may be designed to process up to 100 transactions per second; an operating system may be designed to handle up to 200 separate terminals. Tests have to be designed to ensure that the system can process its intended load. This usually involves planning a series of tests where the load is steadily increased. Stress testing continues these tests beyond the maximum design load of the system until the system fails. Stress testing is particularly relevant to distributed systems based on a network of processors. These systems often exhibit severe degradation when they are heavily loaded as the network becomes swamped with data, which the different processes must exchange.

___ for each phase are established and "___" for unit-tested modules are defined.

Start and end dates, availability windows

Steps involved in back-to-back testing are:

Step 1: prepare a general-purpose set of test case. Step 2: run one version of the program with these test cases and save the results in more than one files Step 3: run another version of the program with the same test cases, saving the results to a different file. Step 4: automatically compare the files produced by the modified and unmodified program versions.

An overall plan for integration of the software and a description of specific tests are documented in a ___.

Test Specification

Like all other elements of a software configuration, the ___ may be tailored to the local needs of a software development organization.

Test Specification format

___ does provide the last bastion from which quality can be assessed and, more pragmatically, errors can be uncovered.

Testing

___ is bounded, criteria for completion of each test phase are described, and schedule constraints are documented.

Testing effort

Managing Testing Functions

Testing is a set of activities that can be planned in advance and conducted systematically. For this reason a template for software testing- a set of steps into which we call in place specific test case design techniques and testing methods- should be defined for the software engineering process. A number of software testing strategies have been proposed in the literature. All provide the software developer with a template for testing and all have the following generic characteristics: Testing begins at the module level and works "outward" toward the integration of the entire computer-based system. Different testing techniques are appropriate at different points in time. Testing is conducted by the developer of the software and (for large projects) an independent test group. Testing and debugging are different activities, but debugging must be accommodated in any testing strategy. Test management tools are used to control and coordinate software testing for each of the major testing steps. Tools in this category manage and co-ordinate regression testing, perform comparisons that ascertain differences between actual and expected output, and conduct batch testing of programs with interactive human-computer interfaces. In addition to the functions noted above, many test management tools also serve as generic test drivers. A test driver reads one or more test cases from a testing file, formats the test data to conform to the needs of the software under test and then invokes the software to be tested. Testing tools in this sub category are customized by the tester to meet specialized testing needs. Finally, test managers sometimes work in conjunction with requirements tracing tools to provide requirements coverage analysis for testing. Reading each test case in sequence, the requirement coverage analyzer attempts to determine (based on information that describes the purpose of the test case) which software requirements are addressed by the test. A cross reference matrix is often used to indicate which tests address, what requirements

Acceptance Testing

Testing is usually relied on to detect the faults, in addition to the faults introduced during coding phase itself. Due to this, different levels of testing are used in testing process; each level of testing aims to test different aspects of the system.

Thread testing

Thread testing is a testing strategy, which was devised for testing real-time systems. It is an event-based approach where tests are based on the events, which trigger system actions. A comparable approach may be used to test object-oriented systems as they may be modeled as event driven systems. Thread testing is a testing strategy, which may be used after processes, or objects have been individually tested and integrated in to sub-systems. The processing of each possible external event „threads‟ its way through the system processes or objects with some processing carried out at each stage. Thread testing involves identifying and executing each possible processing „thread‟. Of course, complete thread testing may be impossible because of the number of possible input and output combinations. In such cases, the most commonly exercised threads should be identified and selected for testing.

Top-Down Testing

Top-down testing tests the high levels of a system before testing its detailed components. The program is represented as a single abstract component with sub components represented by stubs. Stubs have the same interface as the component but very limited functionality After the top-level component has been tested, its stub components are implemented and tested in the same way. This process continues recursively until the bottom level components are implemented. The whole system may then be completely tested.Top-down testing should be used with top-down program development so that a system component is tested as soon as it is coded. Coding and testing are a single activity with no separate component or module-testing phase. If top-down testing is used unnoticed design errors might be detected at an early stage in the testing process. As these errors are usually structural errors, early detection means that they can be corrected without undue costs. Early error detection means that extensive redesign and re-implementation may be avoided. Top-down testing has the further advantage that a limited, working system is available at an early stage in the development. This is an important psychological boost to those involved in the system development. It demonstrates the feasibility of the system to management. Validation, as distinct from verification, can begin early in the testing process as a demonstrable system can be made available to users.

Testing plays an extremely important role in ___.

Verification and validation

___ encompasses a wide array of ___ that include formal technical reviews, quality and configuration audits, performance monitoring, simulation, feasibility study, documentation review, database review, algorithm analysis, development testing, qualification testing, and installation testing.

Verification and validation, SQA activities

Boehm states this in another way:

Verification: "Are we building the product right?" Validation: "Are we building the right product?"

White box testing (glass-box testing)

White-box testing of software is predicated on close examination of procedural detail. Providing test cases that exercise specific sets of conditions and/or loops tests logical paths through the software. The "status of the program" may be examined at various points to determine if the expected or asserted status corresponds to the actual status. White-box testing, sometimes called glass-box testing is a test case design method that uses the control structure of the procedural design to derive test cases.

If top-down development is combined with bottom-up testing, ___.

all parts of the system must be implemented before testing can begin

Software validation is achieved through a series of ___ that demonstrate conformity with requirements.

black box tests

The specification is deliverable in the ___ process and becomes part of the ___.

software engineering, software configuration

The definition of V&V encompasses many of the activities that we have referred to as ___.

software quality assurance (SQA)

A _ process including testing is an impractical approach, particularly if existing software components are to be reused.

strict top-down development

A ___ outlines the classes of tests to be conducted and a ___ defines specific test cases that will be used to demonstrate conformity with requirements.

test plan, test procedure

Verification refers to the set of activities that ensure ___.

that software correctly implements a specific function

Validation refers to a different set of activities that ensure ___.

that the software that has been built is traceable to customer requirements