Chapter 8: Testing
beta testing
A release of the software is made available to users to allow them to experiment and to raise problems that they discover with the system developers.
Types of user testing
Alpha testing, Beta testing, Acceptance testing
procedural interfaces
Sub-system encapsulates a set of procedures to be called by other sub-systems.
______ is intended to show that a program does what it is intended to do and to discover program defects before it is put into use.
Testing
Alpha testing
Users of the software work with the development team to test the software at the developer's site.
_____ is testing where you expect the system to perform correctly using a given set of test cases that reflect the system's expected use.
Validation testing (positive thinking)
is "Are we building the right product" a form of verification or validation?
Validation; software should do what user really requires
Is "Are we building the project right" a form of verification or validation?
Verification; software should conform to its specification
Simplified debugging
When a test fails, it should be obvious where the problem lies. The newly written code needs to be checked and modified.
What was Test-driven development introduced as?
a part of agile methods (such as extreme programming); can be used in plan-driven development processes
Test-driven development (TDD)
an approach to program development in which you inter-leave testing and code development.
When you test software, you execute a program using _______
artificial data
Whenever possible, unit testing should be _____________
automated so that tests are run and checked without manual intervention
Tests are written before _____________
code
_____ testing is where several individual units are integrated to create composite components. This testing should focus on testing _______ interfaces
component
Interface testing guidelines
design tests so that parameters to a called procedure are at the extreme ends of their ranges; always test pointer parameters with null pointers; design tests which cause the component to fail; use stress testing in message passing systems; in shared memory systems, vary the order in which components are activated
_____ testing is where the system is tested during development to discover bugs and defects.
development
Because inspection is a static process, you _____ have to be concerned with interactions between errors. (do or do not)
do not
_____ verification is software testing concerned with exercising and observing product behavior
dynamic
True or false: manually creating testing algorithms is preferred over automatic testing algorithms
false
True or false: User testing is not important if release testing has already been done
false, user testing should always be done no matter the circumstances
Why is user testing essential?
influences from the user's working environment have a major effect on the reliability, performance, usability and robustness of a system. These cannot be replicated in a testing environment
Software _____ involve people examining the source representation with the aim of discovering anomalies and defects.
inspections
Objectives are to detect faults due to find errors or invalid assumptions about the _____
interface
User or customer testing
is a stage in the testing process in which users or customers provide input and advice on system testing
Interface types
parameter interfaces, shared memory interfaces, procedural interfaces, message passing interfaces
What are the testing strategies?
partition testing, guideline-based testing
_____ testing is testing the system to check that changes have not 'broken' previously working code.
regression
_____ testing is the process of testing a particular release of a system that is intended for use outside of the development team.
release
_____ testing is where a separate testing team test a complete version of the system before it is released to users.
release
Software inspections may be applied to any representation of the system like
requirements, design, configuration data, test data, etc
What can program testing do?
reveal the presence of errors NOT their absence
In an automatic test algorithm, the _____ part initializes the system with the test case, the _____ part that calls the object to be tested, and the _____ part that verifies if the output is correct
setup, call, assertion
Confidence depends on what?
software purpose, user expectations, and marketing environment
_____ verification is software inspections concerned with analysis of the static system representation to discover problems
static
Testing is part of a more general verification and validation process, which also includes ____________
static validation techniques
_____ testing is a form of performance testing where the system is deliberately overloaded to test its failure behavior.
stress
_____ testing is where some or all of the components in a system are integrated and the system is tested as a whole.
system
What must happen for the change to be committed in regression testing?
tests must run successfully
What is the aim of V & V?
to establish confidence that the system is 'fit for purpose'
In regression testing, all tests are run again whenever a change is made to the code
true
True or false: Errors can hide other errors
true
True or false: When testing, put in sequences of one or zero length to see results
true
_____ testing is where individual program units or object classes are tested.
unit
_____ testing is where users or potential users of a system test the system in their own environment.
user
Partition testing
where you identify groups of inputs that have common characteristics and should be processed in the same way
Guideline-based testing
where you use testing guidelines to choose test cases, these guidelines reflect previous experience of the kinds of error that are often made when developing components
Interface errors
~Interface misuse, ~Interface misunderstanding, ~Timing errors.
message passing interfaces
§Sub-systems request services from other sub-systems
Important differences between release testing and system testing
-A separate team that has not been involved in the system development, should be responsible for release testing. -System testing by the development team should focus on discovering bugs in the system (defect testing). The objective of release testing is to check that the system meets its requirements and is good enough for external use (validation testing).
Examples of testing policies
-All system functions that are accessed through menus should be tested. -Combinations of functions (e.g. text formatting) that are accessed through the same menu must be tested. -Where user input is provided, all functions must be tested with both correct and incorrect input
Advantages of inspections
-Because inspection is a static process, you don't have to be concerned with interactions between errors -Incomplete versions of a system can be inspected without additional costs -inspection can also consider broader quality attributes of a program, such as compliance with standards, portability and maintainability.
Stages in the acceptance testing process
-Define acceptance criteria -Plan acceptance testing -Derive acceptance tests -Run acceptance tests -Negotiate test results -Reject/accept system
General testing guidelines
-Test with sequences of zero length. -Choose inputs that force the system to generate all error messages -Design inputs that cause input buffers to overflow -Repeat the same input or series of inputs numerous times -Force invalid outputs to be generated -Force computation results to be too large or too small.
Complete test coverage of a class involves ____________________
-Testing all operations associated with an object -Setting and interrogating all object attributes -Exercising the object in all possible states.
What are the 2 types of unit test case?
-The first of these should reflect normal operation of a program and should show that the component works as expected. -The other kind of test case should be based on testing experience of where common problems arise. It should use abnormal inputs to check that these are properly processed and do not crash the component.
interface misunderstanding
A calling component embeds assumptions about the behavior of the called component which are incorrect.
interface misuse
A calling component calls another component and makes an error in its use of its interface e.g. parameters in the wrong order.
shared memory interfaces
Block of memory is shared between procedures or functions.
acceptance testing
Customers test a system to decide whether or not it is ready to be accepted from the system developers and deployed in the customer environment. Primarily for custom systems.
parameter interfaces
Data passed from one method or procedure to another.
_____ is testing where the test cases are designed to expose defects. The test cases in defect testing can be deliberately obscure and need not reflect how the system is normally used.
Defect testing (negative thinking)
Code coverage
Every code segment that you write has at least one associated test so all code written has at least one test.
True or false: Release testing should be done by the developers of the software.
False, testing should be done by a group not associated with the software
What is one test automation framework?
JUnit
When creating software, verification and validation confidence is dependant on software purpose, user expectations, and _____
Marketing environment (Explanation: Getting a product to market early may be more important than finding defects in the program.)
Interface testing
Objectives are to detect faults due to interface errors or invalid assumptions about interfaces
timing errors
The called and the calling component operate at different speeds and out-of-date information is accessed
§Software purpose
The level of confidence depends on how critical the software is to an organization.
System documentation
The tests themselves are a form of documentation that describe what the code should be doing.
What is one program testing goal?
To demonstrate to the developer and the customer that the software meets its requirements
What is another program testing goal?
To discover situations in which the behavior of the software is incorrect, undesirable or does not conform to its specification
True or false: Every requirement in the requirements document should be tested at least once.
True