System and Acceptance Testing

Testing Step 4: Create the test oracle

An oracle contains the predicted results for a set of test cases. The test oracle has to be written down prior to actual testing taking place.

Acceptance Testing: Alpha Test

Client uses the software at the developer's environment and the software is used in a controlled setting with the developer ready to fix bugs

Acceptance Testing: Beta Test

Conducted in the client's environment without a developer present. The software gets a realistic workout in the target environment

Functional Testing

Goal: Test functionality of a system -Test cases are designed from the requirements analysis document (better user manual) and centered around requirements and key functions (use cases) -The system is treated as a black box -Unit test cases can be reused, but new test cases have to be developed as well

Acceptance Testing

Goal: demonstrate system is ready for operational use -Choice of tests is made by the client -Many tests can be taken from integration testing -Acceptance test is performed by client and not the developer -Alpha Test -Beta Test

Performance Testing

Goal: try to violate non-functional requirements -Test how the system behaves when overloaded. Can bottlenecks be identified? (First candidates for redesign in the next iteration -Try unusual orders of execution such as calling a receive() before a send() -Check the system response to large volumes of data beyond expected parameters such as trying to handle 1001 items when the system is designed to handle 1000

Stress test

Stress the limits of the system

Compatibility test

Test backward compatibility with existing systems

Testing Step 3: Develop test cases

Test case is a set of test data or situations that will be used to exercise the unit (class, subsystem, system) being tested or about the attribute being measured

Quality test

Test reliability, maintainability, and availability

Recovery test

Test the system response to presence of errors or loss of data

Configuration test

Test the various software and hardware configurations

Timing test

evaluate response times and time to perform a function

Environmental test

test tolerances for heat, humidity, motion, etc.

Volume test

test what happens if large amounts of data are handled

Human factors test

test with end users

security test

try to violate security requirements

Testing Step 1: Select what has to be tested

-Analysis: completeness of requirements -Design: Cohesion -Implementation" Source Code

Testing Step 2: Decide how the testing is done

-Review or code inspection -Proofs -Black box and white box -Select integration testing strategy

Activities of testing

1. Establish test objectives 2. design test cases 3. write test cases 4. test the test cases 5. execute the tests 6. evaluate the test results 7. change the system 8. do regression testing

Three phases of system testing

1. Functional testing: validates functional requirements 2. Performance testing: validates non-functional requirements 3. Acceptance Testing: validates clients expectations

4 Testing Steps

1. Select what has to be tested 2. Decide how testing is done 3. Develop test cases 4. Create the test oracle

Types of performance testing

1. Stress test 2. volume test 3. configuration test 4. compatibility test 5. timing test 6. security test 7. environmental test 8. quality test 9. recovery test 10. Human factors test

The testing team

1. The programmer is to familiar with the code and is excluded from testing 2. Professional tester 3. Analyst 4. User 5. System designer 6. Configuration management specialist

Guidance for Test Case Selection

1. Use analysis knowledge about functional requirements (black box testing), such as use cases, expected input data, and invalid input data 2. Use design knowledge about system structure, algorithms, data structures (white box testing), such as control structures, test branches, loops, data structures, test records fields, arrays. 3. Use implementation Knowledge about algorithms and data structures such as, force a division by zero or if an upper bound of an array is 10 then use an index of 11.