CSCI 387: Software Testing (Ole Miss)

There is no single authority in charge of the system so top-down control is:

impossible.

Complexity arises because different parts of the system are:

independently managed as is the network.

Unit testing

individual program units or object classes are tested; Unit testing should focus on testing the functionality of objects or methods.

System testing

involves integrating components to create a version of the system and then testing the integrated system; The focus in system testing is testing the interactionsbetween components; System testing checks that components are compatible, interact correctly and transfer the right data at the right time across their interfaces; System testing tests the emergent behavior of a system.

Software Inspection

involves people examining the source representation with the aim of discovering anomalies and defects; Inspections do not require execution of a system so may be used before implementation; They may be applied to any representation of the system (requirements, design, configuration data, test data, etc.); They have been shown to be an effective technique for discovering program errors

Code coverage

measures the degree to which the source code of a program has been tested

Test coverage

measures the degree to which the specification or code of a software program has been exercised by tests

Inheritance makes it ________________ to design object class tests as the information to be tested is not localized.

more difficult

User testing

users or potential users of a system test the system in their own environment.

Control flow testing

uses the control structure of a program to develop the test cases for the program; The test cases are developed to sufficiently cover the whole control structure of the program; The control structure of a program can be represented by the control flow graph of the program.

What can a unit be?

Individual functions or methods within an object, Object classes with several attributes and methods, Composite components with defined interfaces used to access their functionality.

Can Inspections check non-functional characteristics such as performance, usability, etc?

No

Advantages of Inspections

Errors can mask (hide) other errors: because inspection is a static process, you don't have to be concerned with interactions between errors; Incomplete versions of a system can be inspected without additional costs. If a program is incomplete, then you need to develop specialized test harnesses to test the parts that are available; An inspection can also consider broader quality attributes of a program, such as compliance with standards, portability and maintainability.

Condition coverage

Every condition takes TRUE and FALSE outcomes at least once in some test case

Edge coverage

Every edge (branch) of the control flow is traversed at least once by some test case

testing policies

Exhaustive system testing is impossible so testing policies which define the required system test coverage may be developed

Path coverage

Finds the number of distinct paths through the program to be traversed at least once

Marketing environment expectation

Getting a product to market early may be more important than finding defects in the program.

Verification

"Are we building the product right": The software should conform to its specification.

Validation

"Are we building the right product": The software should do what the user really requires.

Aim of Verification & Validation is to establish confidence that the system is:

'fit for purpose'.

2 types of unit test case:

1) test case should reflect normal operation of a program and should show that the component works as expected. 2) test case should be based on testing experience of where common problems arise. It should use abnormal inputs to check that these are properly processed and do not crash the component.

Interface misuse

A calling component calls another component and makes an error in its use of its interface e.g. parameters in the wrong order.

Interface misunderstanding

A calling component embeds assumptions about the behavior of the called component which are incorrect.

When should tests be written during the software development process?

Before the code is written, the testing driving the direction of the development

Shared memory interfaces

Block of memory is shared between procedures or functions.

Which should be used during the Verification and Validation process: Inspection or Testing?

Both

Software inspections

Concerned with analysis of the static system representation to discover problems (static verification) May be supplement by tool-based document and code analysis.

Software testing

Concerned with exercising and observing product behavior (dynamic verification)The system is executed with test data and its operational behavior is observed.

Parameter interfaces

Data passed from one method or procedure to another.

Statement coverage

Each statement executed at least once by some test case

Procedural interfaces

Sub-system encapsulates a set of procedures to be called by other sub-systems; Message passing interfaces Sub-systems request services from other sub-systems

defect testing

Test cases are designed to expose defects; Used to discover faults or defects in the software where its behavior is incorrect or not in conformance with its specification; A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system

When should unit testing be automated?

Whenever possible

Complete test coverage of a class involves:

Testing all operations associated with an object, Setting and interrogating all object attributes, Exercising the object in all possible states.

Timing errors

The called and the calling component operate at different speeds and out-of-date information is accessed.

Software purpose expectations

The level of confidence depends on how critical the software is to an organisation.

Which type of development testing is a defect testing process?

Unit testing

User expectations

Users may have low expectations of certain kinds of software.

validation testing

Uses a given set of test cases that reflect the system's expected use; Used to demonstrate to the developer and the system customer that the software meets its requirements; A successful test shows that the system operates as intended.

state of an object

a constraint on the values of object's attributes; Because the methods use the attributes in computing the object's behavior, the behavior depends on the object state

Release testing

a separate testing team test a complete version of the system before it is released to users.

Interface Testing

aims to detect faults due to interface errors or invalid assumptions about interfaces.

Test-driven development (TDD)

an approach to program development in which you inter-leave testing and code development.

Call par

automated test component that calls the object or method to be tested.

Assertion part

automated test component that compares the result of the call with the expected result. If the assertion evaluates to true, the test has been successful if false, then it has failed.

Setup part

automated test component that initializes the system with the test case, namely the inputs and expected outputs.

use-cases developed to identify system interactions _____ be used as a basis for system testing

can

Inspections and testing are _____________ and not ______________ verification techniques.

complementary, opposing

Distributed systems are more ________ than systems that run on a single processor.

complex

Logical organization of testing

component code to unit testing --> tested component to integration testing --> integrated model to system testing --> function, quality, acceptance, and installation testing

State-based testing

defines a set of abstract states that a software unit can take and tests the unit's behavior by comparing its actual states to the expected states; This approach has become popular with object-oriented systems

Component testing

several individual units are integrated to create composite components; Component testing should focus on testing component interfaces.

System testing

some or all of the components in a system are integrated and the system is tested as a whole; System testing should focus on testing component interactions.

Inspections can check conformance with a ______________ but not conformance with the ____________ ____________ ______________.

specification, customer's real requirements.

Partition testing

testing strategy that identifies groups of inputs that have common characteristics and should be processed in the same way

Guideline-based testing

testing strategy that uses testing guidelines that reflect previous experience of the kinds of errors that programmers often make to choose test cases.

Development testing

the system is tested during development to discover bugs and defects.