Types of Testing
Top-down Integration Testing pros/cons
Advantages: 1.Fault Localization is easier. 2.Possibility to obtain an early prototype. 3.Critical Modules are tested on priority; major design flaws could be found and fixed first. Disadvantages: 1.Needs many Stubs. 2.Modules at a lower level are tested inadequately.
Bottom-up Integration Testing pros/cons
Advantages: 1.Fault localization is easier. 2.No time is wasted waiting for all modules to be developed unlike Big-bang approach Disadvantages: 1.Critical modules (at the top level of software architecture) which control the flow of application are tested last and may be prone to defects. 2.An early prototype is not possible
Example of Integration Test Case
Test Case ID - 1 Test Case Objective - Check the interface link between the Login and Mailbox module Test Case Description - Enter login credentials and click on the Login button Expected Result - To be directed to the Mail Box
Which Test Cases to Automate? High Risk - Business Critical test cases
Test cases to be automated can be selected using the following criterion to increase the automation ROI
Which Test Cases to Automate? Test Cases that are very tedious or difficult to perform manually
Test cases to be automated can be selected using the following criterion to increase the automation ROI
Which Test Cases to Automate? Test Cases which are time-consuming
Test cases to be automated can be selected using the following criterion to increase the automation ROI
Which Test Cases to Automate? Test cases that are repeatedly executed
Test cases to be automated can be selected using the following criterion to increase the automation ROI
Which Test Cases to Automate? Test Cases for which the requirements are frequently changing
The following category of test cases are not suitable for automation:
Which Test Cases to Automate? Test Cases that are newly designed and not executed manually at least once
The following category of test cases are not suitable for automation:
Which Test Cases to Automate? Test cases which are executed on an ad-hoc basis.
The following category of test cases are not suitable for automation:
Difference Between Manual Testing and Automation Testing Initial investment
The initial investment in the automated testing is higher. Though the ROI is better in the long run. The initial investment in the Manual testing is comparatively lower. ROI is lower compared to Automation testing in the long run.
Big Bang Testing
Big Bang Testing(Тестирование большого взрыва) is an Integration testing approach in which all the components or modules are integrated together at once and then tested as a unit. This combined set of components is considered as an entity while testing. If all of the components in the unit are not completed, the integration process will not execute. Advantages: Convenient for small systems.
System test falls under the ... testing category of software testing.
Black Box Testing
Difference Between Manual Testing and Automation Testing Engagement
Done by tools. Its accurate and never gets bored! Repetitive Manual Test Execution can get boring and error-prone.
Difference between Re-Testing and Regression Testing:
Retesting means testing the functionality or bug again to ensure the code is fixed. If it is not fixed, Defect needs to be re-opened. If fixed, Defect is closed. Regression testing means testing your software application when it undergoes a code change to ensure that the new code has not affected other parts of the software.
System Testing
SYSTEM TESTING(системное тестирование) is a level of testing that validates the complete and fully integrated software product.
Smoke Testing Vs Sanity Testing Aim
Smoke Testing is performed to ascertain that the critical functionalities of the program is working fine Sanity Testing is done to check the new functionality/bugs have been fixed
Smoke Testing Vs Sanity Testing subset
Smoke testing is a subset of Acceptance testing Sanity testing is a subset of Regression Testing
Smoke Testing Vs Sanity Testing Documentation
Smoke testing is usually documented or scripted Sanity testing is usually not documented and is unscripted
In SDLC, STLC, V Model, ... testing is first level of testing done
UNIT TESTING(Модульное тестирование)
Unit testing - manual vs automated
Unit testing is commonly automated but may still be performed manually. Software Engineering does not favor one over the other but automation is preferred. A manual approach to unit testing may employ a step-by-step instructional document. Under the automated approach- A developer writes a section of code in the application just to test the function. They would later comment out and finally remove the test code when the application is deployed.
Developers generally use ... framework to develop automated test cases for unit testing.
UnitTest framework
An excellent example of non-functional test
how many people can simultaneously login into a software.
Integration Test Case differs from other test cases in
in the sense it focuses mainly on the interfaces & flow of data/information between the modules. Here priority is to be given for the integrating links rather than the unit functions which are already tested. Sample Integration Test Cases for the following scenario: Application has 3 modules say 'Login Page', 'Mailbox' and 'Delete emails' and each of them is integrated logically. Here do not concentrate much on the Login Page testing as it's already been done in Unit Testing. But check how it's linked to the Mail Box Page. Similarly Mail Box: Check its integration to the Delete Mails Module.
automation ROI meaning
return on investment ROI, or return on investment, is a metric to calculate the efficiency of any investment. In our case, we'll discuss test automation. ... Though automation testing requires more initial cost, the cost will reduce as the process progresses. Automation testing has a lot of advantages over manual testing.
The goal of Automation is
to reduce the number of test cases to be run manually and not to eliminate Manual Testing altogether.
Example Test Cases Non-Functional Testing
1 Application load time should not be more than 5 secs up to 1000 users accessing it simultaneously Performance Testing 2 Software should be installable on all versions of Windows and Mac Compatibility Testing 3 All web images should have alt tags Accessibility testing.
Unit Testing Disadvantages
1.Unit testing can't be expected to catch every error in a program. It is not possible to evaluate all execution paths even in the most trivial programs 2.Unit testing by its very nature focuses on a unit of code. Hence it can't catch integration errors or broad system level errors.
Smoke Testing
In Smoke Testing(дымовое тестирование), the test cases chose to cover the most important functionality or component of the system. The objective is not to perform exhaustive testing, but to verify that the critical functionalities of the system are working fine. For Example, a typical smoke test would be - Verify that the application launches successfully, Check that the GUI is responsive ... etc. The main purpose of smoke testing is to reject a software application with defects so that QA team does not waste time testing broken software application.
Difference Between Manual Testing and Automation Testing Investment
Investment is required for testing tools as well as automation engineers Investment is needed for human resources.
Manual Testing
Manual Testing/Ручное тестирование is a type of software testing in which test cases are executed manually by a tester without using any automated tools.
What is Non-Functional Testing?
NON-FUNCTIONAL TESTING(нефункциональное тестирование) is defined as a type of Software testing to check non-functional aspects (performance, usability, reliability, etc) of a software application.
Difference Between Manual Testing and Automation Testing Cost-effective
Not cost effective for low volume regression Not cost effective for high volume regression.
What is Regression Testing?
REGRESSION TESTING(регрессивное тестирование) is defined as a type of software testing to confirm that a recent program or code change has not adversely affected existing features. Regression Testing is nothing but a full or partial selection of already executed test cases which are re-executed to ensure existing functionalities work fine.
Regression Test Selection
Regression Test Selection is a technique in which some selected test cases from test suite are executed to test whether the modified code affects the software application or not. Test cases are categorized into two parts, reusable test cases which can be used in further regression cycles and obsolete test cases which can not be used in succeeding cycles.
Difference Between Manual Testing and Automation Testing Parallel Execution
This testing can be executed on different operating platforms in parallel and reduce test execution time. Manual tests can be executed in parallel but would need to increase your human resource which is expensive
Smoke Testing Vs Sanity Testing Actors
This testing is performed by the developers or testers Sanity testing in software testing is usually performed by testers
Top-down Integration Testing
Top Down Integration Testing(интеграционное тестирование сверху вниз) is a method in which integration testing takes place from top to bottom following the control flow of software system. The higher level modules are tested first and then lower level modules are tested and integrated in order to check the software functionality. Stubs(функция-заглушка) are used for testing if some modules are not ready
White box testing vs Black box testing
White box testing is the testing of the internal workings or code of a software application. In contrast, black box or System Testing is the opposite. System test involves the external workings of the software from the user's perspective.
Difference Between Manual Testing and Automation Testing Test Report Visibility
With automation testing, all stakeholders can login into the automation system and check test execution results Manual Tests are usually recorded in an Excel or Word, and test results are not readily/ readily available.
Difference Between Manual Testing and Automation Testing Batch testing
You can Batch multiple Test Scripts for nightly execution. Manual tests cannot be batched.
Sandwich Testing
a strategy in which top level modules are tested with lower level modules at the same time lower modules are integrated with top modules and tested as a system. It is a combination of Top-down and Bottom-up approaches therefore it is called Hybrid Integration Testing. It makes use of both stubs as well as drivers.
Test Automation Maintenance Approach
an automation testing phase carried out to test whether the new functionalities added to the software are working fine or not.
Acceptance testing
beta testing of the product done by the actual end users.
Integration Testing
defined as a type of testing where software modules are integrated logically and tested as a group.
Define the scope of Automation
1.The features that are important for the business 2.Scenarios which have a large amount of data 3.Common functionalities across applications 4.Technical feasibility 5.The extent to which business components are reused 6.The complexity of test cases 7.Ability to use the same test cases for cross-browser testing The scope of automation is the area of your Application Under Test which will be automated. Following points help determine scope:
Although each software module is unit tested, defects still exist for various reasons like
1.A Module, in general, is designed by an individual software developer whose understanding and programming logic may differ from other programmers. Integration Testing becomes necessary to verify the software modules work in unity 2.At the time of module development, there are wide chances of change in requirements by the clients. These new requirements may not be unit tested and hence system integration Testing becomes necessary. 3.Interfaces of the software modules with the database could be erroneous 4.External Hardware interfaces, if any, could be erroneous 5.Inadequate exception handling could cause issues.
Myths of Manual Testing
1.Anyone can do manual testing 2.Testing ensures 100% Defect free product 3.Automated testing is more powerful than manual testing 4.Testing is easy
Planning, Design, and Development
1.Automation tools selected 2.Framework design and its features 3.In-Scope and Out-of-scope items of automation 4.Automation testbed preparation 5.Schedule and Timeline of scripting and execution 6.Deliverables of Automation Testing During this phase, you create an Automation strategy & plan, which contains the following details-
Approaches, Strategies, Methodologies of Integration Testing
1.Big Bang Approach : 2.Incremental Approach(инкрементное тестирование): which is further divided into the following 1)Top Down Approach 2)Bottom Up Approach 3)Sandwich Approach - Combination of Top Down and Bottom Up
Two Categories of Software Testing
1.Black Box Testing 2.White Box Testing
Types of Manual Testing:
1.Black Box Testing/тестирование черного ящика 2.White Box Testing/тестирование белого ящика 3.Unit Testing/модульное тестирование 4.System Testing/системное тестирование 5.Integration Testing/интеграционное тестирование 6.Acceptance Testing/приёмочное тестирование In fact, any type of software testing type can be executed both manually as well using an automation tool.
Regression Testing and Configuration Management
1.Code being regression tested should be under a configuration management tool 2.No changes must be allowed to code, during the regression test phase. Regression test code must be kept immune to developer changes. 3.The database used for regression testing must be isolated. No database changes must be allowed
four types of frameworks used in automation software testing
1.Data Driven Automation Framework 2.Keyword Driven Automation Framework 3.Modular Automation Framework 4.Hybrid Automation Framework
Unit Testing Advantage
1.Developers looking to learn what functionality is provided by a unit and how to use it can look at the unit tests to gain a basic understanding of the unit API. 2Unit testing allows the programmer to refactor code at a later date, and make sure the module still works correctly (i.e. Regression testing). The procedure is to write test cases for all functions and methods so that whenever a change causes a fault, it can be quickly identified and fixed. 3.Due to the modular nature of the unit testing, we can test parts of the project without waiting for others to be completed.
Big Bang Testing disadvantages
1.Fault Localization is difficult. 2.Given the sheer number of interfaces that need to be tested in this approach, some interfaces link to be tested could be missed easily. 3.Since the Integration testing can commence only after "all" the modules are designed, the testing team will have less time for execution in the testing phase. 4.Since all modules are tested at once, high-risk critical modules are not isolated and tested on priority. Peripheral modules which deal with user interfaces are also not isolated and tested on priority.
How to do Integration Testing?
1.Prepare the Integration Tests Plan 2.Design the Test Scenarios, Cases, and Scripts. 3.Executing the test Cases followed by reporting the defects. 4.Tracking & re-testing the defects. 5.Steps 3 and 4 are repeated until the completion of Integration is successful.
How to perform Manual Testing
1.Read and understand the software project documentation/guides. Also, study the Application Under Test (AUT) if available. 2.Draft Test cases that cover all the requirements mentioned in the documentation. 3.Review and baseline the test cases with Team Lead, Client (as applicable) 4.Execute the test cases on the AUT 5.Report bugs. 6.Once bugs are fixed, again execute the failing test cases to verify they pass
Regression Testing can be carried out using the following techniques
1.Retest All 2.Regression Test Selection 3.Prioritization of Test Cases
Selecting test cases for regression testing
1.Test cases which have frequent defects 2.Functionalities which are more visible to the users 3.Test cases which verify core features of the product 4.Test cases of Functionalities which has undergone more and recent changes 5.All Integration Test Cases 6.All Complex Test Cases 7.Boundary value test cases 8.A sample of Successful test cases 9.A sample of Failure test cases
System Testing involves testing the software code for following
1.Testing the fully integrated applications including external peripherals in order to check how components interact with one another and with the system as a whole. This is also called End to End testing scenario. 2.Verify thorough testing of every input in the application to check for desired outputs. 3.Testing of the user's experience with the application. That is a very basic description of what is involved in system testing. You need to build detailed test cases and test suites that test each aspect of the application as seen from the outside without looking at the actual source code.
Test Driven Development (TDD) & Unit Testing
1.Tests are written before the code 2.Rely heavily on testing frameworks 3.All classes in the applications are tested 4.Quick and easy integration is made possible Programmers think that Integration Testing will catch all errors and do not execute the unit test. Once units are integrated, very simple errors which could have very easily found and fixed in unit tested take a very long time to be traced and fixed.
What Types of System Testing Should Testers Use?
1.Who the tester works for - This is a major factor in determining the types of system testing a tester will use. Methods used by large companies are different than that used by medium and small companies. 2.Time available for testing - Ultimately, all 50 testing types could be used. Time is often what limits us to using only the types that are most relevant for the software project.
the major testing problems for doing regression testing:
1.With successive regression runs, test suites become fairly large. Due to time and budget constraints, the entire regression test suite cannot be executed 2.Minimizing the test suite while achieving maximum Test coverage remains a challenge 3.Determination of frequency of Regression Tests, i.e., after every modification or every build update or after a bunch of bug fixes, is a challenge.
Test tool selection
1/Test Tool selection largely depends on the technology the Application Under Test is built on. For instance, QTP does not support Informatica. So QTP cannot be used for testing Informatica applications. It's a good idea to conduct a Proof of Concept of Tool on AUT.
One of the Software Testing Fundamentals about Automation is
100% Automation is not possible
Framework for Automation
A framework is set of automation guidelines which help in 1.Minimum usage of code 2.Less Maintenance of code 3.Improve re-usability 4.Non Technical testers can be involved in code 5.The training period of using the tool can be reduced 6.Involves Data wherever appropriate
Smoke Testing Vs Sanity Testing priority
As per the needs of testing, you may have to execute both Sanity and Smoke Tests in the software build. In such cases, you will first execute Smoke tests and then go ahead with Sanity Testing. In industry, test cases for Sanity Testing are commonly combined with that for smoke tests, to speed up test execution. Hence, it's a common that the terms are often confused and used interchangeably
Difference Between Manual Testing and Automation Testing When to Use?
Automated Testing is suited for Regression Testing, Performance Testing, Load Testing or highly repeatable functional test cases. Manual Testing is suitable for Exploratory, Usability and Adhoc Testing(Свободное или Интуитивное тестирование). It should also be used where the AUT changes frequently.
Difference Between Manual Testing and Automation Testing Documentation
Automated Tests acts as a document provides training value especially for automated unit test cases. A new developer can look into a unit test cases and understand the code base quickly. Manual Test cases provide no training value
Difference Between Manual Testing and Automation Testing Deadlines
Automated Tests have zero risks of missing out a pre-decided test. Manual Testing has a higher risk of missing out the pre-decided test deadline.
Difference Between Manual Testing and Automation Testing Devops
Automated Tests help in Build Verification Testing(Тестирование сборки) and are an integral part of DevOps Cycle Manual Testing defeats the automated build principle of DevOps
Difference Between Manual Testing and Automation Testing Test Design
Automated Unit Tests enforce/drive Test Driven Development Design.(Разработка через тестирование) Manual Unit Tests do not drive design into the coding process
Difference Between Manual Testing and Automation Testing Human observation
Automated testing does not involve human consideration. So it can never give assurance of user-friendliness and positive customer experience. The manual testing method allows human observation, which may be useful to offer user-friendly system.
Difference Between Manual Testing and Automation Testing Reliability
Automated testing is a reliable method, as it is performed by tools and scripts. There is no testing Fatigue. Manual testing is not as accurate because of the possibility of the human errors.
Difference Between Manual Testing and Automation Testing Processing time
Automated testing is significantly faster than a manual approach. Manual testing is time-consuming and takes up human resources.
Test Execution
Automation Scripts are executed during this phase. The scripts need input test data before there are set to run. Once executed they provide detailed test reports.
Automation Testing
Automation Testing or Test Automation(Автоматизированное тестирование) is a software testing technique that performs using special automated testing software tools to execute a test case suite.
Difference Between Manual Testing and Automation Testing Definition
Automation Testing uses automation tools to execute test cases. In manual testing, test cases are executed by a human tester and software.
Difference Between Manual Testing and Automation Testing Exploratory Testing
Automation does not allow random testing Exploratory testing(исследовательское тестирование) is possible in Manual Testing
Difference Between Manual Testing and Automation Testing Set up
Automation test requires less complex test execution set up. Manual testing needs have a more straightforward test execution setup
Difference Between Manual Testing and Automation Testing Build Verification Testing
Automation testing is useful for Build Verification Testing (BVT)(Тестирование сборки). Executing the Build Verification Testing (BVT) is very difficult and time-consuming in manual testing
Difference Between Manual Testing and Automation Testing Ideal approach
Automation testing is useful when frequently executing the same set of test cases Manual testing proves useful when the test case only needs to run once or twice.
Difference Between Manual Testing and Automation Testing Framework
Automation testing uses frameworks like Data Drive, Keyword, Hybrid to accelerate the automation process. Manual Testing does not use frameworks but may use guidelines, checklists, stringent processes to draft certain test cases.
Bottom-up Integration Testing
Bottom-up Integration Testing(Интегрированное тестирование снизу вверх) is a strategy in which the lower level modules are tested first. These tested modules are then further used to facilitate the testing of higher level modules. The process continues until all modules at top level are tested. Once the lower level modules are tested and integrated, then the next level of modules are formed
Entry and Exit Criteria of Integration Testing
Entry Criteria: 1.Unit Tested Components/Modules 2.All High prioritized bugs fixed and closed 3.All Modules to be code completed and integrated successfully. 4.Integration tests Plan, test case, scenarios to be signed off and documented. 5.Required Test Environment to be set up for Integration testing Exit Criteria: 1.Successful Testing of Integrated Application. 2.Executed Test Cases are documented 3.All High prioritized bugs fixed and closed 4.Technical documents to be submitted followed by release Notes.
Difference Between Manual Testing and Automation Testing UI Change
For even a trivial change in the UI of the AUT, Automated Test Scripts need to be modified to work as expected Small changes like change in id, class, etc. of a button wouldn't thwart execution of a manual tester.
How to do Unit Testing
In order to do Unit Testing, developers write a section of code to test a specific function in software application. Developers can also isolate this function to test more rigorously which reveals unnecessary dependencies between function being tested and other units so the dependencies can be eliminated.
Incremental Testing
In the Incremental Testing approach, testing is done by integrating two or more modules that are logically related to each other and then tested for proper functioning of the application. Then the other related modules are integrated incrementally and the process continues until all the logically related modules are integrated and tested successfully Incremental Approach, in turn, is carried out by two different Methods: 1.Bottom Up 2.Top Down
Difference Between Manual Testing and Automation Testing Performance Testing
Performance Tests like Load Testing, Stress Testing, Spike Testing, etc. have to be tested by an automation tool compulsorily. Performance Testing is not feasible manually
Prioritization of Test Cases
Prioritize the test cases depending on business impact, critical & frequently used functionalities. Selection of test cases based on priority will greatly reduce the regression test suite.
Difference Between Manual Testing and Automation Testing Programming knowledge
Programming knowledge is a must in automation testing. No need for programming in Manual Testing.
What is Sanity Testing?
Sanity testing(санитарное тестирование) is a kind of Software Testing performed after receiving a software build, with minor changes in code, or functionality, to ascertain that the bugs have been fixed and no further issues are introduced due to these changes. The goal is to determine that the proposed functionality works roughly as expected. If sanity test fails, the build is rejected to save the time and costs involved in a more rigorous testing. The objective is "not" to verify thoroughly the new functionality but to determine that the developer has applied some rationality (sanity) while producing the software. For instance, if your scientific calculator gives the result of 2 + 2 =5! Then, there is no point testing the advanced functionalities like sin 30 + cos 50.
Automated Testing Process:
Step 1) Test Tool Selection Step 2) Define scope of Automation Step 3) Planning, Design and Development Step 4) Test Execution Step 5) Maintenance
Stubs and Drivers
Stubs and Drivers are the dummy programs in Integration testing used to facilitate the software testing activity. These programs act as a substitutes for the missing models in the testing. They do not implement the entire programming logic of the software module but they simulate data communication with the calling module while testing. Stub: Is called by the Module under Test. Driver: Calls the Module to be tested.
Smoke Testing Vs Sanity Testing Object of verification
The objective of this testing is to verify the "stability" of the system in order to proceed with more rigorous testing The objective of the testing is to verify the "rationality" of the system in order to proceed with more rigorous testing
Retest All
This is one of the methods for Regression Testing in which all the tests in the existing test bucket or suite should be re-executed. This is very expensive as it requires huge time and resources.
Mock Objects
Unit testing relies on mock objects being created to test sections of code that are not yet part of a complete application. Mock objects fill in for the missing parts of the program. For example, you might have a function that needs variables or objects that are not created yet. In unit testing, those will be accounted for in the form of mock objects created solely for the purpose of the unit testing done on that section of code.
