ISTQB - Automation - 1.1 Purpose of Test Automation

Testware is necessary for the testing activities that include: list them.

1)*Implementing* automated test cases 2)*Monitoring and controlling* the execution of automated tests 3)*Interpreting, reporting and logging* the automated test results

Disadvantages of test automation

1)Additional costs are involved 2)Initial investment to setup TAS 3)Requires additional technologies 4)Team needs to have development and automation skills 5)On-going TAS maintenance requirement 6)Can distract from testing objectives, e.g., focusing on automating tests cases at the expense of executing tests 7)Tests can become more complex 8)Additional errors may be introduced by automation

Objectives of test automation

1)Improving test efficiency 2)Providing wider function coverage 3)Reducing the total test cost 4)Performing tests that manual testers cannot 5)Shortening the test execution period 6)Increasing the test frequency/reducing the time required for test cycles

Advantages of test automation

1)More tests can be run per build 2)The possibility to create tests that cannot be done manually (real-time, remote, parallel tests) 3)Tests can be more complex 4)Tests run faster 5)Tests are less subject to operator error 6)More effective and efficient use of testing resources 7)Quicker feedback regarding software quality 8)Improved system reliability (e.g., repeatability, consistency) 9)Improved consistency of tests

Limitations of test automation

1)Not all manual tests can be automated 2)The automation can only check machine-interpretable results 3)The automation can only check actual results that can be verified by an automated test oracle 4)Not a replacement for exploratory testing

Test automation has different approaches for interacting with a SUT. List them.

1)Testing through the *public interfaces* to classes, modules or libraries of the SUT (*API testing*) 2)Testing through the *user interface* of the SUT (e.g., *GUI testing or CLI testing*) 3)Testing through a *service or protocol*

List tasks of test automation

1)Using purpose *built software tools* to control and set up test preconditions 2)Executing tests 3)Comparing actual outcomes to predicted outcomes

What is from below is test automation task activitiy approach objective advantage disadvantage limitation? 1.Additional costs are involved 2.Additional errors may be introduced by automation 3.Can distract from testing objectives, e.g., focusing on automating tests cases at the expense of executing tests 4.Comparing actual outcomes to predicted outcomes 5.Executing tests 6.Implementing automated test cases 7.Improved consistency of tests 8.Improved system reliability (e.g., repeatability, consistency) 9.Improving test efficiency 10.Increasing the test frequency 11.Initial investment to setup TAS 12.Interpreting, reporting and logging the automated test results 13.Monitoring and controlling the execution of automated tests 14.More effective and efficient use of testing resources 15.More tests can be run per build 16.Not a replacement for exploratory testing 17.Not all manual tests can be automated 18.On-going TAS maintenance requirement 19.Performing tests that manual testers cannot 20.Providing wider function coverage 21.Quicker feedback regarding software quality 22.Reducing the time required for test cycles 23.Reducing the total test cost 24.Requires additional technologies 25.Shortening the test execution period 26.Team needs to have development and automation skills 27.Testing through a service or protocol 28.Testing through the public interfaces to classes, modules or libraries of the SUT (API testing) 29.Testing through the user interface of the SUT (e.g., GUI testing or CLI testing) 30.Tests are less subject to operator error 31.Tests can be more complex 32.Tests can become more complex 33.Tests run faster 34.The automation can only check actual results that can be verified by an automated test oracle 35.The automation can only check machine-interpretable results 36.The possibility to create tests that cannot be done manually (real-time, remote, parallel tests) 37.Using purpose built software tools to control and set up test preconditions

1.Additional costs are involved - disadvantage 2.Additional errors may be introduced by automation - disadvantage 3.Can distract from testing objectives, e.g., focusing on automating tests cases at the expense of executing tests - disadvantage 4.Comparing actual outcomes to predicted outcomes - task 5.Executing tests - task 6.Implementing automated test cases - activitiy 7.Improved consistency of tests - advantage 8.Improved system reliability (e.g., repeatability, consistency) - advantage 9.Improving test efficiency - objective 10.Increasing the test frequency - objective 11.Initial investment to setup TAS - disadvantage 12.Interpreting, reporting and logging the automated test results - activity 13.Monitoring and controlling the execution of automated tests - activitiy 14.More effective and efficient use of testing resources - advantage 15.More tests can be run per build - advantage 16.Not a replacement for exploratory testing - limitation 17.Not all manual tests can be automated - limitation 18.On-going TAS maintenance requirement - disadvantage 19.Performing tests that manual testers cannot - objective 20.Providing wider function coverage - objective 21.Quicker feedback regarding software quality - advantage 22.reducing the time required for test cycles - objective 23.Reducing the total test cost - objective 24.Requires additional technologies - disadvantage 25.Shortening the test execution period - objective 26.Team needs to have development and automation skills - disadvantage 27.Testing through a service or protocol - approach 28.Testing through the public interfaces to classes, modules or libraries of the SUT (API testing) - approach 29.Testing through the user interface of the SUT (e.g., GUI testing or CLI testing) - approach 30.Tests are less subject to operator error - advantage 31.Tests can be more complex - advantage 32.Tests can become more complex - disadvantage 33.Tests run faster - advantage 34.The automation can only check actual results that can be verified by an automated test oracle - limitation 35.The automation can only check machine-interpretable results - limitation 36.The possibility to create tests that cannot be done manually (real-time, remote, parallel tests) - advantage 37.Using purpose built software tools to control and set up test preconditions - task

Is test automation *only* a mechanism for running a test suite without human interaction?

No. It involves a process of designing the testware.

List what does automation testware include

Software Documentation Test cases Test environments Test data

It is a good practice to separate the software used for testing from the system under test (SUT) itself. Are there any exceptions?

There are exceptions, for example *embedded systems* where the test software needs to be deployed to the SUT.

Why is it a good practice is to separate the software used for testing from the system under test (SUT) itself?

to *minimize interference*.