ISTQB CTFL - Section 3

Types of Reviews

A single software product or related work product may be the subject of more than one review. If more than one type of review is used, the order may vary.

Entry Criteria

Entry Criteria is the set of generic and specific conditions for permitting a process to go forward with a defined task, e.g. test phase. The purpose of Entry Criteria is to prevent a task from starting which would entail more wasted effort compared to the effort needed to remove the failed entry criteria.

Benefits of reviews

Benefits of reviews include: 1. early defect detection and correction, 2. development productivity improvements, 3. reduced development timescales, 4. reduced testing cost and time, 5. lifetime cost reductions, 6. fewer defects and improved communication. Reviews can find omissions, for example, in requirements, which are unlikely to be found in dynamic testing.

Compiler

Compiler is a software tool that translates programs expressed in a high-order language into their machine-language equivalents.

Complexity

Complexity is the degree to which a component or system has a design and/or internal structure that is difficult to understand, maintain, and verify.

Control Flow

Control Flow is a sequence of events or paths in the execution through a component or system.

Data Flow

Data Flow is an abstract representation of the sequence and possible changes of state and data objects, where the state of an object is any of creation, usage, or destruction.

Dynamic Testing

Dynamic testing is testing that involves the execution of the software of a component or system. Dynamic testing can show failures that are caused by defects.

Exit Criteria

Entry Criteria is the set of generic and specific conditions, agreed upon by the stakeholders for permitting a process to be officially completed. The purpose of Exit Criteria is to prevent a task from being considered completed when there are oustanding parts of the task which have not been completed. Exit Criteria are used to report against and to plan when to stop testing. Typically exit criteria may cover the following: 1. Thoroughness measures, such as coverage of code, functionality or risk, 2. Estimates of defect density or reliability measures, 3. Cost, 4. Residual risks such as defects not fixed or lack of test coverage in certain areas, 5. Schedules such as those based on time to market

Formal Review

Formal Review is a review characterized by documented procedures and requirements, e.g. inspection.

Informal Review

Informal Review is a review not based on a formal (documented) procedure. It may take the form of pair programming or a technical lead reviewing designs and code. Main purpose: inexpensive way to get some benefit.

Inspection

Inspection is a type of peer review that relies on visual examination of documents to detect defects, e.g. violations of development standards and non-conformance to higher-level documentation. An inspection is the most formal review technique and therefore is always based on a documented procedure. It is led by trained moderator and includes metrics gathering as well as specified entry and exit criteria for acceptance of the software product. Main purpose: finding defects

Checklists

Looking at software products or related work products from different perspectives and using checklists can make reviews more effective and efficient. For example, a checklist based on various perspectives such as user, maintainer, tester or operations, or a checklist of typical requirements problems may help to uncover previously undetected issues.

Metric

Metric is a measurement scale and the method used for measurement.

Moderator

Moderator is the leader and main person responsible for an inspection or other review process. A Moderator can also be a neutral person who conducts a usability test session.

Peer Review

Peer Review is a review of a software work product by colleagues of the producer of the product for the purpose of identifying defects and improvements. Examples are inspection, technical review, and walkthrough.

Reviewer

Reviewer is the person involved in the review that identifies and describes anomalies in the product or project under review. Reviewers can be chosen to represent different viewpoints and roles in the review process.

Reviews

Reviews are a way of testing software work products (including code) and can be performed well before dynamic test execution. Defects detected during reviews early in the life cycle (e.g., defects found in requirements) are often much cheaper to remove than those detected by running tests on the executing code.

Scribe

Scribe is the person who records each defect mentioned and any suggestions for process improvement during a review meeting, on a logging form. The scribe should ensure that the logging form is readable and understandable.

Static Analysis

Static Analysis is an analysis of a software artifact, e.g. requirements or code, carried out without execution of those software artifacts. Static Analysis is usually carried out by means of a supporting tool.

Static Testing

Static Testing is testing of a component or system at specification or implementation level without execution of the software, e.g. reviews or static analysis.

Success Factors for Reviews

Success factors for reviews include: 1. Each review has clear predefined objectives, 2. the right people for the review objectives are involved, 3. Testers are valued reviewers who contribute to the review and also learn about the product which enables them to prepare tests earlier, 4. Defects found are welcomed and expressed objectively, 5. People issues and psychological aspects are dealt with, 6. the review is conducted in an atmosphere of trust; 7. the outcome will not be used for the evaluation of the participants, 8. Review techniques are applied that are suitable to achieve the objectives and to the type and level of software work products and reviewers, 9. Checklists or roles are used if appropriate to increase effectiveness of defect identification, 10. Training is given in review techniques, especially the more formal techniques such as inspection, 11. Management supports a good review process, 12. there is an emphasis on learning and process improvement

Technical Review

Technical Review is a peer group discussion activity that focuses on achieving consensus on the technical approach to be taken. It is a documented, defined defect-detection process that includes peers and technical experts with optional management participation. The preparation of a review report which includes the list of findings, the verdict whether the software product meets its requirements and, where appropriate, recommendations related to findings. Main purposes: discussing, making decisions, evaluating alternatives, finding defects, solving technical problems and checking conformance to specifications, plans, regulations, and standards. See also Peer Review.

Roles in a formal review: Author

The Author the writer or person with chief responsibility for the document(s) to be reviewed.

Roles in a formal review: Manager

The Manager decides on the execution of reviews, allocates time in project schedules and determines if the review objectives have been met.

Roles in a formal review: Moderator

The Moderator is the person who leads the review of the document or set of documents, including planning the review, running the meeting, and following-up after the meeting. If necessary, the moderator may mediate between the various points of view and is often the person upon whom the success of the review rests.

Roles in a formal review: Reviewers

The Reviewers are individuals with a specific technical or business background (also called checkers or inspectors) who, after the necessary preparation, identify and describe findings (e.g., defects) in the product under review. Reviewers should be chosen to represent different perspectives and roles in the review process, and should take part in any review meetings.

Roles in a formal review: Scribe or Recorder

The Scribe documents all the issues, problems and open points that were identified during the meeting.

Activities of a Formal Review

The activities of a formal review include: 1. Planning, 2. Kick-off, 3. Individual Preparation, 4. Examination/Evaluation/Recording of Results/Review Meeting, 5. Rework, 6. Follow-up

The Objective of Static Analysis

The objective of static analysis is to find defects in software source code and software models. Static analysis is performed without actually executing the software being examined by the tool; dynamic testing does execute the software code. Static analysis can locate defects that are hard to find in dynamic testing. As with reviews, static analysis finds defects rather than failures. Static analysis tools analyze program code (e.g., control flow and data flow), as well as generated output such as HTML and XML.

The Value of Static Analysis

The value of Static Analysis is: 1. Early detection of defects prior to test execution, 2. Early warning about suspicious aspects of the code or design by the calculation of metrics, such as a high complexity measure, 3. Identification of defects not easily found by dynamic testing, 4. Detecting dependencies and inconsistencies in software models such as links, 5. Improved maintainability of code and design, 6. Prevention of defects, if lessons are learned in development

Typical Defects discovered by Static Analysis

Typical defects discovered by static analysis tools include: 1. Referencing a variable with an undefined value, 2. Inconsistent interfaces between modules and components, 3. Variables that are not used or are improperly declared, 4. Unreachable or dead code, 5. Missing and erroneous logic (potentially infinite loops), 6. Overly complicated constructs, 7. Programming standards violations, 8. Security vulnerabilities, 9. Syntax violations of code and software models

Defects found in reviews

Typical defects that are easier to find in reviews than in dynamic testing include: 1.deviations from standards, 2. requirement defects, 3. design defects, 4. insufficient maintainability and 5. incorrect interface specifications.

Walkthrough

Walkthrough is a step-by-step presentation by the author of a document in order to gather information and to establish a common understanding of its content. It may take the form of scenarios, dry runs, peer group participation. Main purposes: learning, gaining understanding, finding defects. See also Peer Review.