System and Acceptance Testing
Testing Step 4: Create the test oracle
An oracle contains the predicted results for a set of test cases. The test oracle has to be written down prior to actual testing taking place.
Acceptance Testing: Alpha Test
Client uses the software at the developer's environment and the software is used in a controlled setting with the developer ready to fix bugs
Acceptance Testing: Beta Test
Conducted in the client's environment without a developer present. The software gets a realistic workout in the target environment
Functional Testing
Goal: Test functionality of a system -Test cases are designed from the requirements analysis document (better user manual) and centered around requirements and key functions (use cases) -The system is treated as a black box -Unit test cases can be reused, but new test cases have to be developed as well
Acceptance Testing
Goal: demonstrate system is ready for operational use -Choice of tests is made by the client -Many tests can be taken from integration testing -Acceptance test is performed by client and not the developer -Alpha Test -Beta Test
Performance Testing
Goal: try to violate non-functional requirements -Test how the system behaves when overloaded. Can bottlenecks be identified? (First candidates for redesign in the next iteration -Try unusual orders of execution such as calling a receive() before a send() -Check the system response to large volumes of data beyond expected parameters such as trying to handle 1001 items when the system is designed to handle 1000
Stress test
Stress the limits of the system
Compatibility test
Test backward compatibility with existing systems
Testing Step 3: Develop test cases
Test case is a set of test data or situations that will be used to exercise the unit (class, subsystem, system) being tested or about the attribute being measured
Quality test
Test reliability, maintainability, and availability
Recovery test
Test the system response to presence of errors or loss of data
Configuration test
Test the various software and hardware configurations
Timing test
evaluate response times and time to perform a function
Environmental test
test tolerances for heat, humidity, motion, etc.
Volume test
test what happens if large amounts of data are handled
Human factors test
test with end users
security test
try to violate security requirements
Testing Step 1: Select what has to be tested
-Analysis: completeness of requirements -Design: Cohesion -Implementation" Source Code
Testing Step 2: Decide how the testing is done
-Review or code inspection -Proofs -Black box and white box -Select integration testing strategy
Activities of testing
1. Establish test objectives 2. design test cases 3. write test cases 4. test the test cases 5. execute the tests 6. evaluate the test results 7. change the system 8. do regression testing
Three phases of system testing
1. Functional testing: validates functional requirements 2. Performance testing: validates non-functional requirements 3. Acceptance Testing: validates clients expectations
4 Testing Steps
1. Select what has to be tested 2. Decide how testing is done 3. Develop test cases 4. Create the test oracle
Types of performance testing
1. Stress test 2. volume test 3. configuration test 4. compatibility test 5. timing test 6. security test 7. environmental test 8. quality test 9. recovery test 10. Human factors test
The testing team
1. The programmer is to familiar with the code and is excluded from testing 2. Professional tester 3. Analyst 4. User 5. System designer 6. Configuration management specialist
Guidance for Test Case Selection
1. Use analysis knowledge about functional requirements (black box testing), such as use cases, expected input data, and invalid input data 2. Use design knowledge about system structure, algorithms, data structures (white box testing), such as control structures, test branches, loops, data structures, test records fields, arrays. 3. Use implementation Knowledge about algorithms and data structures such as, force a division by zero or if an upper bound of an array is 10 then use an index of 11.