Chapter 8 - Software Testing
What happens in *Software Inspections*
- involves people *examining* the source representation with the aim of *discovering* anomalies and defects. - *do not require* execution of a system so may be used before implementation. (Static Testing) - may be applied to *any representation* of the system (requirements, design,configuration data, test data, etc.).
Performance Testing: *Stress testing*
- is a form of performance testing where the system is deliberately overloaded to test its failure behavior.
What is *User Testing*
- is a stage in the testing process in which users or customers provide input and advice on system testing. - is essential, even when comprehensive system and release testing have been carried out.
What is *Test-driven development (TDD)*
- is an approach to program development in which you inter-leave testing and code development. - Tests are written before code and 'passing' the tests is the critical driver of development. - You develop code incrementally, along with a test for that increment. You don't move on to the next increment until the code that you have developed passes its test.
What is *Program Testing*
- is intended to show that a *program does* what it is intended to do and to discover program *defects* before it is put into *use*. - is part of a more general *verification and validation* process.
What is *Release testing*
- is the process of *testing a particular release* of a system that is intended for use *outside of the development team*. - *primary goal* is to *convince the supplier* of the system that it is *good enough for use*. - usually a *black-box testing* process where tests are only derived from the system specification.
Development Testing: *Unit Testing*
- is the process of testing individual components in isolation. - is a defect testing process.
What *Inspections* Can and Cannot Do
- *can check* conformance with a *specification* but not conformance with the customer's *real requirements*. - *cannot check non-functional characteristics* such as performance, usability, etc.
TDD Benefit: Regression testing
- A regression test suite is developed incrementally as a program is developed. - is testing the system to check that changes have not 'broken' previously working code. - Tests must run 'successfully' before the change is committed.
What is *V & V Confidence*
- Aim of V & V is to establish confidence that the system is *'fit for purpose'*. - Depends on software purpose, user expectations and marketing environment
Testing Policies: Examples
- All system functions that are *accessed through menus* should be tested. - *Combinations of functions* (e.g. text formatting) that are accessed through the same menu must be tested. - Where *user input is provided*, all functions must be tested with both correct and incorrect input.
Unit Testing: Examples of Units
Units may be: - Individual functions or methods within an object. - Object classes with several attributes and methods - Composite components with defined interfaces used to access their functionality.
Types of User Testing: Beta testing
- A release of the software is made available to users to allow them to experiment and to raise problems that they discover with the system developers.
What is *Software Inspection*
- Concerned with analysis of the static system representation to discover problems (static verification) -- May be supplement by tool-based document and code analysis.
What is *Software Testing*
- Concerned with exercising and observing product behavior (dynamic verification) -- The system is executed with test data and its operational behavior is observed.
Types of User Testing: Acceptance testing
- Customers test a system to decide whether or not it is ready to be accepted from the system developers and deployed in the customer environment. Primarily for custom systems.
Acceptance testing: Stages in Process
- Define acceptance criteria - Plan acceptance testing - Derive acceptance tests - Run acceptance tests - Negotiate test results - Reject/accept system
What are some advantages of *Software Inspections*
- During testing, *errors can mask (hide) other errors*. Because inspection is a static process, you don't have to be concerned with interactions between errors. - *Incomplete versions* of a system can be inspected without additional costs. - As well as searching for program defects, an inspection can also consider broader *quality attributes* of a program, such as compliance with standards, portability and maintainability.
TDD Benefit: Code coverage
- Every code segment that you write has at least one associated test so all code written has at least one test.
Why have *Testing policies*
- Exhaustive system testing is impossible so testing policies which define the required system test coverage may be developed.
V & V Confidence: Marketing environment
- Getting a product to market early may be more important than finding defects in the program.
Interface Testing
- Objectives are to detect faults due to interface errors or invalid assumptions about interfaces.
Release Testing VS System Testing
- Release testing is a form of system testing. Important Differences: - *A separate team* that has *not been involved* in the system development, should be responsible for release testing. - System testing by the development team should focus on discovering bugs in the system *(defect testing)*. The objective of release testing is to check that the system meets its requirements and is good enough for *external use (validation testing)*.
Development Testing: *Component Testing*
- Software components are often composite components that are made up of *several interacting objects*. - Testing composite components should therefore focus on showing that the *component interface* behaves according to its specification.
V & V Confidence: Software purpose
- The level of confidence depends on how critical the software is to an organization.
What is *Verification*
- The software should conform to its specification. - i.e. "Are we building the product right".
What is *Validation*
- The software should do what the user really requires. - i.e. "Are we building the right product".
TDD Benefit: System documentation
- The tests themselves are a form of documentation that describe what the code should be doing.
What are *Program Testing Goals*
- To *demonstrate* to the developer and the customer that the software meets its requirements. - To *discover* situations in which the behavior of the software is incorrect, undesirable or does not conform to its specification.
What is *Validation Testing*
- To demonstrate to the developer and the system customer that the software *meets its requirements* - A successful test shows that the system operates as intended.
What is *Defect Testing*
- To discover *faults or defects* in the software where its behavior is incorrect or not in conformance with its specification. - A successful test is a test that makes the system *perform incorrectly* and so exposes a defect in the system.
V & V Confidence: User expectations
- Users may have low expectations of certain kinds of software.
Types of User Testing: Alpha testing
- Users of the software work with the development team to test the software at the developer's site.
TDD Benefit: Simplified debugging
- When a test fails, it should be obvious where the problem lies. The newly written code needs to be checked and modified.
What happens during *Program Testing*
- When you *test software*, you execute a program using *artificial data*(test data). - You check the results of the test run for *errors, anomalies* or information about the program's *non-functional attributes*.
Why use *Automated Testing*
- Whenever possible, unit testing should be automated so that tests are run and checked without manual intervention. - you make use of a test automation framework (such as JUnit) to write and run your program tests.
Inspections and Testing
- are *complementary* and not opposing verification techniques. - Both should be used during the V & V process
What is *System Testing*
- involves *integrating components* to create a version of the system and then *testing the integrated system*. - focus is testing the *interactions between components*. - checks that components are *compatible, interact correctly* and *transfer the right data* at the right time across their interfaces.
What is *Requirements based testing*
- involves examining each requirement and developing a test or tests for it.
What happens during *System and Component Testing*
- reusable components that have been separately developed and *off-the-shelf systems* may be integrated with newly developed components. The complete system is then tested. - Components *developed by different team members or sub-teams* may be integrated at this stage. System testing is a collective rather than an individual process.
What is *Performance testing*
- usually involves planning a series of tests where the load is steadily increased until the system performance becomes unacceptable.
What is *Development Testing*
- where the system is tested *during development* to discover bugs and defects. - includes all testing activities that are carried out by the team developing the system.
What are (3) *Types of User Testing*
1. Alpha testing 2. Beta testing 3. Acceptance testing
What are the (4) Benefits of *Test-driven development*
1. Code coverage 2. Regression testing 3. Simplified debugging 4. System documentation
What are the (3) *Stages of Testing*
1. Development testing 2. Release testing 3. User testing
Interface Testing: *Interface Types*
1. Parameter interfaces - Data passed from one method or procedure to another. 2. Shared memory interfaces - Block of memory is shared between procedures or functions. 3. Procedural interfaces - Sub-system encapsulates a set of procedures to be called by other sub-systems. 4. Message passing -interfaces Sub-systems request services from other sub-systems
Development Testing: *Testing Activities*
1. Unit testing 2. Component testing 3. System testing
What are the *Testing Process Goals* (2)
1. Validation Testing 2. Defect Testing
Automated Test Components
1. setup part - where you initialize the system with the test case, namely the inputs and expected outputs. 2. call part - where you call the object or method to be tested. 3. assertion part - where you compare the result of the call with the expected result. If the assertion evaluates to true, the test has been successful if false, then it has failed.