Chapter 8: Testing

beta testing

A release of the software is made available to users to allow them to experiment and to raise problems that they discover with the system developers.

Types of user testing

Alpha testing, Beta testing, Acceptance testing

procedural interfaces

Sub-system encapsulates a set of procedures to be called by other sub-systems.

______ is intended to show that a program does what it is intended to do and to discover program defects before it is put into use.

Testing

Alpha testing

Users of the software work with the development team to test the software at the developer's site.

_____ is testing where you expect the system to perform correctly using a given set of test cases that reflect the system's expected use.

Validation testing (positive thinking)

is "Are we building the right product" a form of verification or validation?

Validation; software should do what user really requires

Is "Are we building the project right" a form of verification or validation?

Verification; software should conform to its specification

Simplified debugging

When a test fails, it should be obvious where the problem lies. The newly written code needs to be checked and modified.

What was Test-driven development introduced as?

a part of agile methods (such as extreme programming); can be used in plan-driven development processes

Test-driven development (TDD)

an approach to program development in which you inter-leave testing and code development.

When you test software, you execute a program using _______

artificial data

Whenever possible, unit testing should be _____________

automated so that tests are run and checked without manual intervention

Tests are written before _____________

code

_____ testing is where several individual units are integrated to create composite components. This testing should focus on testing _______ interfaces

component

Interface testing guidelines

design tests so that parameters to a called procedure are at the extreme ends of their ranges; always test pointer parameters with null pointers; design tests which cause the component to fail; use stress testing in message passing systems; in shared memory systems, vary the order in which components are activated

_____ testing is where the system is tested during development to discover bugs and defects.

development

Because inspection is a static process, you _____ have to be concerned with interactions between errors. (do or do not)

do not

_____ verification is software testing concerned with exercising and observing product behavior

dynamic

True or false: manually creating testing algorithms is preferred over automatic testing algorithms

false

True or false: User testing is not important if release testing has already been done

false, user testing should always be done no matter the circumstances

Why is user testing essential?

influences from the user's working environment have a major effect on the reliability, performance, usability and robustness of a system. These cannot be replicated in a testing environment

Software _____ involve people examining the source representation with the aim of discovering anomalies and defects.

inspections

Objectives are to detect faults due to find errors or invalid assumptions about the _____

interface

User or customer testing

is a stage in the testing process in which users or customers provide input and advice on system testing

Interface types

parameter interfaces, shared memory interfaces, procedural interfaces, message passing interfaces

What are the testing strategies?

partition testing, guideline-based testing

_____ testing is testing the system to check that changes have not 'broken' previously working code.

regression

_____ testing is the process of testing a particular release of a system that is intended for use outside of the development team.

release

_____ testing is where a separate testing team test a complete version of the system before it is released to users.

release

Software inspections may be applied to any representation of the system like

requirements, design, configuration data, test data, etc

What can program testing do?

reveal the presence of errors NOT their absence

In an automatic test algorithm, the _____ part initializes the system with the test case, the _____ part that calls the object to be tested, and the _____ part that verifies if the output is correct

setup, call, assertion

Confidence depends on what?

software purpose, user expectations, and marketing environment

_____ verification is software inspections concerned with analysis of the static system representation to discover problems

static

Testing is part of a more general verification and validation process, which also includes ____________

static validation techniques

_____ testing is a form of performance testing where the system is deliberately overloaded to test its failure behavior.

stress

_____ testing is where some or all of the components in a system are integrated and the system is tested as a whole.

system

What must happen for the change to be committed in regression testing?

tests must run successfully

What is the aim of V & V?

to establish confidence that the system is 'fit for purpose'

In regression testing, all tests are run again whenever a change is made to the code

true

True or false: Errors can hide other errors

true

True or false: When testing, put in sequences of one or zero length to see results

true

_____ testing is where individual program units or object classes are tested.

unit

_____ testing is where users or potential users of a system test the system in their own environment.

user

Partition testing

where you identify groups of inputs that have common characteristics and should be processed in the same way

Guideline-based testing

where you use testing guidelines to choose test cases, these guidelines reflect previous experience of the kinds of error that are often made when developing components

Interface errors

~Interface misuse, ~Interface misunderstanding, ~Timing errors.

message passing interfaces

§Sub-systems request services from other sub-systems

Important differences between release testing and system testing

-A separate team that has not been involved in the system development, should be responsible for release testing. -System testing by the development team should focus on discovering bugs in the system (defect testing). The objective of release testing is to check that the system meets its requirements and is good enough for external use (validation testing).

Examples of testing policies

-All system functions that are accessed through menus should be tested. -Combinations of functions (e.g. text formatting) that are accessed through the same menu must be tested. -Where user input is provided, all functions must be tested with both correct and incorrect input

Advantages of inspections

-Because inspection is a static process, you don't have to be concerned with interactions between errors -Incomplete versions of a system can be inspected without additional costs -inspection can also consider broader quality attributes of a program, such as compliance with standards, portability and maintainability.

Stages in the acceptance testing process

-Define acceptance criteria -Plan acceptance testing -Derive acceptance tests -Run acceptance tests -Negotiate test results -Reject/accept system

General testing guidelines

-Test with sequences of zero length. -Choose inputs that force the system to generate all error messages -Design inputs that cause input buffers to overflow -Repeat the same input or series of inputs numerous times -Force invalid outputs to be generated -Force computation results to be too large or too small.

Complete test coverage of a class involves ____________________

-Testing all operations associated with an object -Setting and interrogating all object attributes -Exercising the object in all possible states.

What are the 2 types of unit test case?

-The first of these should reflect normal operation of a program and should show that the component works as expected. -The other kind of test case should be based on testing experience of where common problems arise. It should use abnormal inputs to check that these are properly processed and do not crash the component.

interface misunderstanding

A calling component embeds assumptions about the behavior of the called component which are incorrect.

interface misuse

A calling component calls another component and makes an error in its use of its interface e.g. parameters in the wrong order.

shared memory interfaces

Block of memory is shared between procedures or functions.

acceptance testing

Customers test a system to decide whether or not it is ready to be accepted from the system developers and deployed in the customer environment. Primarily for custom systems.

parameter interfaces

Data passed from one method or procedure to another.

_____ is testing where the test cases are designed to expose defects. The test cases in defect testing can be deliberately obscure and need not reflect how the system is normally used.

Defect testing (negative thinking)

Code coverage

Every code segment that you write has at least one associated test so all code written has at least one test.

True or false: Release testing should be done by the developers of the software.

False, testing should be done by a group not associated with the software

What is one test automation framework?

JUnit

When creating software, verification and validation confidence is dependant on software purpose, user expectations, and _____

Marketing environment (Explanation: Getting a product to market early may be more important than finding defects in the program.)

Interface testing

Objectives are to detect faults due to interface errors or invalid assumptions about interfaces

timing errors

The called and the calling component operate at different speeds and out-of-date information is accessed

§Software purpose

The level of confidence depends on how critical the software is to an organization.

System documentation

The tests themselves are a form of documentation that describe what the code should be doing.

What is one program testing goal?

To demonstrate to the developer and the customer that the software meets its requirements

What is another program testing goal?

To discover situations in which the behavior of the software is incorrect, undesirable or does not conform to its specification

True or false: Every requirement in the requirements document should be tested at least once.

True