CSCI 387: Software Testing (Ole Miss)
There is no single authority in charge of the system so top-down control is:
impossible.
Complexity arises because different parts of the system are:
independently managed as is the network.
Unit testing
individual program units or object classes are tested; Unit testing should focus on testing the functionality of objects or methods.
System testing
involves integrating components to create a version of the system and then testing the integrated system; The focus in system testing is testing the interactionsbetween components; System testing checks that components are compatible, interact correctly and transfer the right data at the right time across their interfaces; System testing tests the emergent behavior of a system.
Software Inspection
involves people examining the source representation with the aim of discovering anomalies and defects; Inspections do not require execution of a system so may be used before implementation; They may be applied to any representation of the system (requirements, design, configuration data, test data, etc.); They have been shown to be an effective technique for discovering program errors
Code coverage
measures the degree to which the source code of a program has been tested
Test coverage
measures the degree to which the specification or code of a software program has been exercised by tests
Inheritance makes it ________________ to design object class tests as the information to be tested is not localized.
more difficult
User testing
users or potential users of a system test the system in their own environment.
Control flow testing
uses the control structure of a program to develop the test cases for the program; The test cases are developed to sufficiently cover the whole control structure of the program; The control structure of a program can be represented by the control flow graph of the program.
What can a unit be?
Individual functions or methods within an object, Object classes with several attributes and methods, Composite components with defined interfaces used to access their functionality.
Can Inspections check non-functional characteristics such as performance, usability, etc?
No
Advantages of Inspections
Errors can mask (hide) other errors: because inspection is a static process, you don't have to be concerned with interactions between errors; Incomplete versions of a system can be inspected without additional costs. If a program is incomplete, then you need to develop specialized test harnesses to test the parts that are available; An inspection can also consider broader quality attributes of a program, such as compliance with standards, portability and maintainability.
Condition coverage
Every condition takes TRUE and FALSE outcomes at least once in some test case
Edge coverage
Every edge (branch) of the control flow is traversed at least once by some test case
testing policies
Exhaustive system testing is impossible so testing policies which define the required system test coverage may be developed
Path coverage
Finds the number of distinct paths through the program to be traversed at least once
Marketing environment expectation
Getting a product to market early may be more important than finding defects in the program.
Verification
"Are we building the product right": The software should conform to its specification.
Validation
"Are we building the right product": The software should do what the user really requires.
Aim of Verification & Validation is to establish confidence that the system is:
'fit for purpose'.
2 types of unit test case:
1) test case should reflect normal operation of a program and should show that the component works as expected. 2) test case should be based on testing experience of where common problems arise. It should use abnormal inputs to check that these are properly processed and do not crash the component.
Interface misuse
A calling component calls another component and makes an error in its use of its interface e.g. parameters in the wrong order.
Interface misunderstanding
A calling component embeds assumptions about the behavior of the called component which are incorrect.
When should tests be written during the software development process?
Before the code is written, the testing driving the direction of the development
Shared memory interfaces
Block of memory is shared between procedures or functions.
Which should be used during the Verification and Validation process: Inspection or Testing?
Both
Software inspections
Concerned with analysis of the static system representation to discover problems (static verification) May be supplement by tool-based document and code analysis.
Software testing
Concerned with exercising and observing product behavior (dynamic verification)The system is executed with test data and its operational behavior is observed.
Parameter interfaces
Data passed from one method or procedure to another.
Statement coverage
Each statement executed at least once by some test case
Procedural interfaces
Sub-system encapsulates a set of procedures to be called by other sub-systems; Message passing interfaces Sub-systems request services from other sub-systems
defect testing
Test cases are designed to expose defects; Used to discover faults or defects in the software where its behavior is incorrect or not in conformance with its specification; A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system
When should unit testing be automated?
Whenever possible
Complete test coverage of a class involves:
Testing all operations associated with an object, Setting and interrogating all object attributes, Exercising the object in all possible states.
Timing errors
The called and the calling component operate at different speeds and out-of-date information is accessed.
Software purpose expectations
The level of confidence depends on how critical the software is to an organisation.
Which type of development testing is a defect testing process?
Unit testing
User expectations
Users may have low expectations of certain kinds of software.
validation testing
Uses a given set of test cases that reflect the system's expected use; Used to demonstrate to the developer and the system customer that the software meets its requirements; A successful test shows that the system operates as intended.
state of an object
a constraint on the values of object's attributes; Because the methods use the attributes in computing the object's behavior, the behavior depends on the object state
Release testing
a separate testing team test a complete version of the system before it is released to users.
Interface Testing
aims to detect faults due to interface errors or invalid assumptions about interfaces.
Test-driven development (TDD)
an approach to program development in which you inter-leave testing and code development.
Call par
automated test component that calls the object or method to be tested.
Assertion part
automated test component that compares the result of the call with the expected result. If the assertion evaluates to true, the test has been successful if false, then it has failed.
Setup part
automated test component that initializes the system with the test case, namely the inputs and expected outputs.
use-cases developed to identify system interactions _____ be used as a basis for system testing
can
Inspections and testing are _____________ and not ______________ verification techniques.
complementary, opposing
Distributed systems are more ________ than systems that run on a single processor.
complex
Logical organization of testing
component code to unit testing --> tested component to integration testing --> integrated model to system testing --> function, quality, acceptance, and installation testing
State-based testing
defines a set of abstract states that a software unit can take and tests the unit's behavior by comparing its actual states to the expected states; This approach has become popular with object-oriented systems
Component testing
several individual units are integrated to create composite components; Component testing should focus on testing component interfaces.
System testing
some or all of the components in a system are integrated and the system is tested as a whole; System testing should focus on testing component interactions.
Inspections can check conformance with a ______________ but not conformance with the ____________ ____________ ______________.
specification, customer's real requirements.
Partition testing
testing strategy that identifies groups of inputs that have common characteristics and should be processed in the same way
Guideline-based testing
testing strategy that uses testing guidelines that reflect previous experience of the kinds of errors that programmers often make to choose test cases.
Development testing
the system is tested during development to discover bugs and defects.