ISTQB Chapter 5 Test Management

What are some factors in project risk?

- organizational factors ( personal, political issues, skill, attitude) - technical issues (problems in defining right requirements, low quality of the design, test environment not ready) - supplier issues (contract issues, 3rd party failure)

What are the benefits of independence?

- see other/different defects and are unbiased - can verify assumptions people made during specification and implementation

Planning is influenced by:

- test policy -scope -objectives -risks -constraints -criticality -testability -availability of resources

Who should test best for large, complex and safety critical projects?

multiple levels or testing with some or all the levels done by independent testers

What is the role of the test leader compared to the tester?

test leader: plans, monitors, and controls the testing activities and tasks tester: testing the component or system

What are the options for independent testing?

1. Developers responsible; no independent testing 2. Independent testers within the development team 3. Independent test team within the organization 4. Independent testers from the business organization or user community 5. Independent test specialists for specific test types ( usability, performance, ...) 6. Independent test specialists external or outsourced to the organization

What are some examples of test control actions?

- Making decisions based on information from test monitoring - Re-prioritize tests when an identified risk occurs (e.g. software delivered late) - Change the test schedule due to availability of a test environment - Set an entry criterion requiring fixes to have been retested by a developer before accepting them into a build.

What are the objectives of incident reports?

- Provide developers and other parties with feedback about the problem to enable identification, isolation and correction as necessary. - Provide test leaders a means of tracking the quality of the system under test and the progress of the testing. - Provide ideas for test process improvement

What are some common test metrics during test monitoring?

- % of work done in test case preparation - % of work done in test environment preparation - test case execution (# of Pass/Fail) -defect density - test coverage - confidence of testers in the product - dates of test milestones -testing costs

During test reporting, metrics collected during and at the end of a test level in order to assess:

- adequacy of the test objectives for that test level - adequacy of the test approaches taken - effectiveness of the testing

Test planning activities include:

- determine scope/risks/objective - define test levels, entry, exit criteria - scheduling test analysis, design activities, test implementation, execution and evaluation - assigning resources -selecting metrics

In risk-based approach the risks identified may be used to:

- determine test techniques to be employed - determine the extent of testing to be carried out - prioritize testing - Determine whether any non-testing activities could be employed to reduce risk

What are some examples of product risks?

- failure-prone software - poor data integrity and quality - software could cause harm to an individual/company - software does not perform intended functions

What are the drawbacks if independence?

- isolation from the development team - developers may lose sense of responsibility for quality - maybe seen as bottleneck/blamed for delays

What does the test plan identify?

-test items - features to be tested - testing tasks -who will do each task - degree of tester indep - test environment - test design techniques -entry/exit criteria - rationale for their choice risks requiring contingency planning

What the typical tasks of the test leader?

1. Coordinate/Write/review the test strategy 2. Plan the tests 3. Initiate the specification, preparation, implementation and execution of tests, monitor the test results and check the exit criteria 4. Adapt planning based on test results and progress 5. Set up adequate configuration management of testware for traceability 6. Introduce suitable metrics for measuring test progress and evaluating the quality of the testing and the product. 7.Decide what should be automated 8. Select tools to support testing 9. Write test summary reports based on the information gathered during testing.

What are the typical tasks of the tester?

1.Review and contribute to test plans. 2. Analyze, review and assess user requirements, specifications and models for testability. 3. Create test specifications. 4. Set up the test environment (often coordinating with system administration and network management). 5. Prepare and acquire test data. 6. Implement tests on all test levels, execute and log tests, evaluate the results and document the deviations from expected results. 7. Automate tests (may be supported by a developer or a test automation expert). 8. Measure performance of components and systems (if applicable). 9. Review tests developed by others.

What is the difference between project risks and product risks?

Project risk: risks that surround the project's capability to deliver its objectives Product risk: potential failure areas in software/system, risk to the quality of the product

What is the risk-based approach to testing?

Provides opportunity to reduce risk in initial stages of project. Identification of product risks and their use in guiding test planning and control, specification, preparation and execution of tests.

What standard does all the outlines come from?

Standard for Software Test Documentation (IEE std 829 - 1998)

What is test reporting?

Summarizing information about: - what happened during a period of testing - analysis of info/metrics to support recommendations/decisions about future actions

What is risk?

the chance of an event, hazard, threat or situation occurring and its undesirable consequences, as a potential problem

Why is the test plan considered a living document?

it relates the test concept to the actual course of the project. It needs to be maintained, adapted and changed according to the projects needs

What are the two basic approaches for test estimation?

metrics-based approach and expert-based approach

What is an incident?

any event uncovered during testing that requires investigation

What is entry criteria?

define when to start testing. (test environment readiness, test tool readiness, testable code/data available)

What is exit criteria?

define when to stop testing. (thoroughness measures, cost, residual risks, estimates of defect density/reliability measures), schedules

What is the test control?

describes any guiding or corrective actions taken as a result of info and metrics gathered and reported. Actions may cover any test activity or may affect any other software life cycle activity

When should the configuration management procedures and tools be chosen, documented, and implemented?

during test planning

What is expert-based approach?

estimating the tasks based on estimates made by the owner of these tasks

What is metrics-based approach?

estimating the testing effort based on metrics of former or similar projects

What is the dynamic and heuristic approach?

ex: exploratory testing. testing is more reactive to events than pre-planned, and where execution and evalution are concurrent tasks

What is the methodical approach?

ex: failure-based (error guessing and fault attacks) experience-based, checklist-based and quality characteristic based

What is the analytical approach?

ex: risk-based testing. where testing is directed to areas of greatest risk

What is the model-based approach?

ex: stochastic testing. use statistical info about failure rates or usage

How does configuration management help the tester?

helps to uniquely identify (and reproduce) the tested item, test documents, tests, and test harness

What is the test approach?

the implementation of the test strategy for a specific project (defined in the test plans and designs). The starting point for planning the test process, selecting test design techniques and test types to be applied, and defining entry/exit criteria

What is a defect density?

the number of confirmed defects detected in software divided by the size of the software

What is the consultative approach?

those in which test coverage is driven primarily by the advice/guidance of technology and/or business domain experts outside the team

What is the process or standard-compliant approach?

those specified by industry-specific standards or the various agile methodologies

What is the regression-averse approach?

those that include reuse of existing test material, extensive automation of functional regression tests, and standard test suites

What is the purpose of configuration management?

to establish and maintain the integrity of the products of the software

What is the purpose of test monitoring?

to provide feedback and visibility about test activities