Testing Fundamentals
Test Closure Deliverables
1) Incident Report (Closed) 2) Testware - Test cases, automated scripts, test data, etc. 3) Tested Software
Test Plan Document (Deliverable for Planning and Control)
A document in which data from all the Planning and control activities are formally documented as per the organizations standards.
Defect
A flaw in the system which arises due to an error committed during design or development activity.
Sanity Tests
A functional test. They are similar to Smoke tests in their objective and their test selection (how quick they can be executed). They differ in terms of their scope and time of execution and are limited to small components and defect fixes.
Smoke Tests
A functional test. Usually are the first tests performed during the test execution phase. Their objective is report whether the software is ready for the full blown testing that will soon follow.
Security Tests
A non-functional test that is conducted to check whether the application allows access to only authorized users and withstands unauthorized attacks.
Software Testing
A set of processes that take place throughout SDLC. It helps reduce the risk of failures that may occur during operation and thus ensure the software system quality.
Error
Also called a mistake, is something that was missed overlooked or implemented incorrectly during designing or developing the sotware
UAT - Alpha Level Testing
Also known as factory acceptance testing. It is done at developing organizations site but not by the development team. Testing is done by users and stakeholders.
UAT - Beta
Also known as field/site acceptance testing and is performed by customers at their own locations.
Static Test Walkthrough
An informal review process. Defects review/comments are communicated orally to the author. Used when an artifact prepared by a team belonging to one discipline needs to be reviewed by a team from another discipline.
System Tests
Another functional test designed from the functional requirements to validate the software. Executed after smoke/sanity tests have reported success. Formal test that tests the bulk of the functional tests during the testing phase.
Static Tests Static Analysis
Can be informal or formal review process based on the person who is performing it. Defects review/comments might be documented if the review if performed by someone other than the author. Used to analyze the dynamic behavior or the artifact without executing it.
Software Defects
Cause loss of money, time, reputation and life. Detecting defects in the early stages of the SDLC is very important.
Environmental conditions
Certain hardware components may be vulnerable to environmental conditions (temperature, pollution, magnetism) which might impact the functionality and performance of the system.
Evaluating exit criteria and reporting
Checking against exit criteria -> Test Results attained are compared with the exit criteria defined in the planning phase. Determining if more tests are required to attain a required quality level. Prepare a report of test summary for stakeholders.
Test Procedure Specification - Implementation and Execution Deliverable
Contains data about the steps to be followed for executing test cases, how testers are going to run the test(Either manually or using automation).
Incident Reporting (Live) - Implementation and Execution Deliverable
Contains information about the incidents(defects) found during testing.
Test Design Specification Document (Deliverable for Analysis and Design Phase)
Deliverable -Document contains the required test items and test conditions.
Implementation and Execution
Development and prioritizing test cases -> (Test Cases are authored using the test conditions, test data designed in previous phase) Setting up Test Environment -> (The environment components figured out in the previous phase is created and put into action here) Execution -> Test cases authored are executed in the environment that has been set up here. Result Comparison -> After execution of test cases whatever results are received will be compared with the expected result that are brought from the test basis document. Report discrepancies -> If the expected and actual result varies, defect or incident has to be reported to the developers. Repeating test activities -> After debugging re-execution or test cases are done in the form of retesting, so as to validate if those features are working.
Root Cause: Code Complexity
Difficulty in analyzing and coding complicated business logic and processing complex data.
Software testing teams
Help in the timely identification of the defects, which help prevent such losses.
Fuctionality
How accurately does it perform the tasks stated and implied in the requirements
Reliability
How capable is it in maintaining its required level of performance over a period of time.
Efficiency
How economically does it consume available hardware resources to maintain the required level of performance of its functions
Visibility
How simple is it for users to understand, learn and operate the software.
Maintainability
How simple is it to analyze the software make changes and test those changes whenever modifications become necessary.
Portability
How simple is it to install, remove or replace the software and how easy is it for other software to integrate with it.
Root Cause: Human Error
Humans often overlook or misinterpret logical errors due to lack of attention to details.
Test Closure
If planned deliverables have been delivered. Recording the acceptance of system by stakeholders. Archiving the test-ware, test environment for later reuse. Handling over the test-ware to maintenance organization. Investigating lessons learned to determine changes needed for the future releases and projects. Helps improving the testing practices and make the process more matured.
Root Cause: Heterogeneous & Interconnected Technologies
In today's world systems interface with many other applications developed in various technologies independently. Interconnection across these systems may cause defects.
Root Cause: Time Crunch
It increases the probability of incomplete deliverables and human errors.
Static Tests Technical Review
It is a formal review process. Defects review/comments are documented, tracked periodically until they are brought to closure. Used when a version of an artifact needs to be finalized based on a review and sign off by a competent authority.
Static Test Technical Review
It is a semi formal review process. Defects review/comments might be communicated orally or documented for further reference and re-review. Used when an artifact prepared by a novice in one discipline needs to be reviewed by an expert from the same discipline.
Accessibility Tests
Non-functional Tests that establish whether individuals with disabilities will be able to operate the system in question.
Performance Tests
Non-functional test that focuses on the response times of each application component are validated to see if they are under acceptable limits.
Fundamental Test Process Stages
Planning and control -> Analysis and Design -> Implementation and execution -> Evaluating exit criteria and reporting -> Test Closure.
Testing
Process done to identify presence of defects and show failures caused by defects. It can be on code as well as other artifacts and is usually done by one tester.
Debugging
Processes done to find, analyze, and remove the cause of failure. It is done only for code and by the developers.
Analysis and Design
Review Test basis - (Test basis documents are what contains your test cases. Issues can be found during the review and the document can be edited) Test Items Identification - (Objects to focus upon while designing tests and preparing test data). Test Condition Identification - (Test condition is an item or event that could be verified by one or more tests) Identifying Relevant Test Data - (Look for the possible set of test data which can be used while executing the test cases) Designing Test Environment - (it is a place where test cases are executed whenever they are prepared in the implementation and execution phase).
Path Coverage Tests
Structural Tests that are aimed at checking all possible program execution flow paths through the program.
Decision Coverage Tests
Structural Tests that are aimed at ensuring that all decision outcomes, in the program are tested at least once.
Statement Coverage Tests
Structural Tests that are aimed at exercising all programming statements with minimal tests.
Non Functional Tests
Tests that are conducted to validate non functional requirements (reliability, usability, efficiency, maintainability, portability). Testing done by specialized testers who possess the skills to analyze and validate non functional characteristics.
Functional Tests
Tests that are conducted to validate the functional requirements (business functionality). Usually done by normal testers.
Structural tests
Tests that are conducted to validate the program code to check whether it conforms to the structure defined in the design. Testing done by developers.
Dynamic Tests
Tests that are performed by running or executing the software code. Dynamic tests are concerned only with the executable programs. Can only be conducted after the coding phase of the SDLC.
Static Tests
Tests that are performed without executing the software code. Almost all the artifacts created throughout the SDLC are capable of being tested.
Regression Tests
Tests that re-run passed test cases after any code change or defect fix, and check if they are still passing to make sure no functionality is broken unintentionally.
Retests
Tests that re-run the failed test cases once their corresponding defects are fixed to confirm if the fix is working. Also known as confirmation testing.
Software Quality
The degree to which a system, component or process conforms to the specified requirements as per user expectations, contractual agreements and regulatory conditions.
Failure
The manifestation of any existing defects in the system when it is executed in real time may lead to what is called a failure. This can be avoided by detecting and correcting defects and preventing errors.
Dynamic Tests White Box Tests
The software is validated by monitoring the code, program, variables, or system resources during execution. There is complete access and visibility into the code. Validations are done from the programmers perspective. The objective is to understand the system, to locate defect or to validate new code/defect fix.
Dynamic Tests Black Box Test
The software is validated using it's user interface. There is no visibility into the underlying source Code. Validations are done from the end users perspective. The objective is to identify the presence of the defects in the underlying code.
Test Case Specification - Implementation and Execution Deliverable
This report will contain information about test cases, environmental needs for those test cases
Test Summary Report - (Evaluating exit criteria and reporting deliverable)
This report will contain the summarized information about requirements, test cases, defects etc. associated to the project.
Component level testing
Used to test the functional and nonfunctional characteristic of the code/module. Done by white box testers
System Level Testing
Used to test the functional and nonfunctional characteristics of the complete system. Done by Black Box testers.
System Integration Level Testing
Used to test the interaction between different systems or between hardware and software. Also done by black box testers.
Component Integration level Testing
Used to test the interaction between software components and is done after component testing. Also done by white box testers.
Planning and Control
You will need to analyze requirements and then determine and document: Testing Objective (What is expected at the end of testing) Testing Scope (What should be tested and what need not be tested) Testing Strategy (How to achieve the test Objective)
The individuals, teams, and organizations building software have the responsibility of:
ensuring quality by 1) Preventing defects from entering in the software 2) Detecting & fixing defects that are existing as early as possible 3) Ensuring compliance with all required standards contractual obligations, and legal regulations.
