Chapter 5 Part 2 Test Progress Monitoring and Control

Enumerate software artifacts that need to be under configuration management

Identification, version control, and version changes.

Recall common metrics used tor test preparation and execution

Information to be monitored may be collected manually or automatically and may be used to measure exit criteria, such as coverage. The information needs to be measurable.

Summarize the content of the test summary report, according to IEEE-829

It's a test summary report identifier, summary, variances, comprehensive assessment, summary of results, evaluation, summary of activities, and approvals.

What is the purpose of Configuration Management?

It's also about mapping what is being tested to the underlying files and components that make it up. This is critical. When we report defects, we need to report which configuration was controlled, or version controlled.

What are typical project risks? Hint: there're three categories.

Organizational factors (shortages of people, skills, training, bad communication, bad attitude and bad expectations), Technical issues (problem defining requirements, constraints, quality of code, design and tests, bad requirements). Supplier issues (problem with platform, hardare, installation, failure to consider issues in the contract, failure to respond to issues and reports).

In risk-based approach, how do you use the risks identified?

- Determine the test techniques to be employed . - Determine the extent of testing to be carried out. - Prioritize testing in an attempt to find the critical defects as early as possible. - Determine whether any non-testing activities could be employed to reduce risk (e.g. providing training to inexperienced designers).

What are some Test Control actions?

- Making decisions based on information from test monitoring - Re-prioritize tests when an identified risk occurs - Change the test schedule due to availability of a test environment. A portion of the software might be delivered late. - Set an entry criterion requiring fixes to have been retested (confirmation tested) by a developer before accepting them into a build. Perhaps a test was cut. Some mistakes might be so severe that you weren't able to proceed until there were corrected.

What is a product risk?

Product risk is directly relayed to the test object. It is the possibility that the system or software might fail to satisfy some reasonable customer, user, or stakeholder expectations. The problem might not even involved functionality, but rather security, reliability, usability, maintainability or performance.

What is a project risk?

A project risk is a risk related to management and control of the test project, e.g. lack of staffing, strict deadlines, changing requirements etc. You need to ask yourself, «What could go wrong on the project to delay or invalidate the test plan, the test strategy and the test estimate? What are unacceptable outcomes of the testing or in testing? What are the likelihoods and impacts of each of these tasks?

What is the objective of an Incident Report?

Provide developers and other parties with feedback about the problem to enable identification, isolation and correction as necessary. Communicate from one person to another, the error.

Explain the concept of risk

Risk is the possibility of a negative or undesirable outcome, the possible problems that might endanger the objectives of the project stakeholders. Might harm the users.

What is Test Reporting?

Test reporting is concerned with summarizing information about the testing endeavor including: - what happening during a period of testing (dates when exit criteria were met), and - Tes status reports are often about enlightening and influencing the stakeholders. This involved analyzing the information and metrics available to support conclusions, recommendations and decisions about how to guide the project forward or to take other actions.

What is risk-based testing?

The idea that we can organize our testing efforts in a way that reduced the residual level of product risk when the system ships. Do it early. It involves both mitigation - testing to provide opportunities to reduce the likelihood of defects, specially high-impact defects - and contingency - testing to identify work-arounds to make the defects that do get past us less painful. With a risk-based approach, using likelihood and impact, we determine the test techniques to be employed, determine the extent of testing to be carried out, prioritize testing in an attempt to find the critical defects as early as possibly, and to determine whether any non-testing activities could be employed to reduce the risk (e.g. provide training to inexperienced designers).

What goes into an incident report?

Description of the situation, screens, steps taken to reproduce incident, impact, classification (scope, severity), level of priority, risks, costs and benefits of fixing/ not fixing, root cause, conclusion and recommendations.

Explain and compare metrics used for test reporting (e.g.: defects found & fixed, tests passed & failed)

The template shows unresolved defects and the percentage of it, planned and actually executed test, and the percentage of executed tests.

What is the difference between an Incident and a Defect?

We sometimes draw a distinctions between incidents on the one hand and defects and bugs on the other. An incident is any situation where the system exhibits questionable behavior, but often we refer to an incident as a defect only when the root cause is some problem in the item we're testing. There cause of incidents include misconfiguration or failure of the test environment, corrupted test data, bad tests, invalid expected results and tester mistakes. Questionable behavior doesn't mean it's a defect.

What are some product risks?

What are some product risks? They can be failure-prone software (unreliable, bad interface), software/hardware could cause harm to an individual or company, poor software characteristics (functionality, reliability, usability and performance), and the software that does not perform its intended functions.