C857: Software Quality Assurance
SDP: Objectives
- "specify what the program should do and how well it should do it" - translate the requirements into specific objectives by assessing feasibility, time, cost, resolving conflicting requirements, and establishing priorities and trade-offs
SDP: Requirements
- "specify why the program is needed" - translate the program user's needs into a set of written requirements - these are the goals for the product
QA proposes a formal group meeting where the programmer narrates, statement by statement, the logic of a program as an error detection technique. Which quality method meets this proposal?
- Code inspection
Which action describes a realistic verification step?
- Compare the input to the system design phase of its input to the program design phase
How does a unit test run on an updated module of code benefit code development?
- Development knows the module meets quality standards and is ready to be integrated.
Inspection Process Steps
- Entry (ready?) - Planning - Kickoff - Checking - Logging - Analyzing - Process Improvement Brainstorming - Edit - Exit
What describes decision/condition coverage testing?
- Exercise all decision outcomes and each condition at least once.
Inspection
- Formal verification technique - work is examined by a group of peers for the explicit purpose of finding defects - led by a moderator; the author is NOT allowed to be the moderator - written action on ALL defects is mandatory - defect data is used to improve the product, the process, and the effectiveness of the inspection process - managers are NOT permitted to attend - collects engineering data - does NOT examine alternatives - verifies work satisfies both specs and preceding work products - verifies work conforms to standards
Which testing process goes along with "SDP: External Specifications"?
- Function Test
Buddy Check
- INFORMAL verification technique - "life-cycle work" product is examined by author and one other person - the author walks the "buddy" through the life-cycle work and describes what is intended in each section - most importantly the buddy decides on the "fit" to the described intent by the author - objectives: -- improve work -- consider alternative approaches -- exchange techniques -- point out problems -- give a different perspective -- mentoring
Which testing process goes along with "SDP: System Design"?
- Integration Test
Which testing process goes along with "SDP: Program Structure Design"?
- Integration Test - Same as SDP: Systems Design step
Why is black box testing considered a useful test design technique?
- It can aid in deriving test data based on analysis of the requirement specifications.
The development team is questioned by its end users concerning the benefit of designing tests in the software development process. What is a key benefit of developing tests early in the process?
- It helps prevent defects from being introduced into the code.
Principal Processes for achieving objective: Progress Check or Evaluation
- Management/Technical Review
Which key process is supported when the developer continuously provides feedback to the QA team after bug reports are delivered?
- Sustaining partnership
Which testing process goes along with "SDP: Objectives"?
- System Test
What is a key testing principle of designing tests?
- Test cases should include invalid and unexpected input conditions.
Which statement describes part of a formal coding review process?
- The programmer narrates, statement by statement the logic of the program.
Technical Review
- a FORMAL team evaluation - identifies any discrepancies from specifications and standards - determines suitability of use - provides recommendations - objective is to make sure life-cycle work conforms to specifications - confirms that work is being done according to plan
What are the necessary parts of a test case, according to the software testing principles?
- a description of the correct output of the program for a set of input data
What is an "external specification"?
- a precise description of the program's behavior from the end-user point of view
Which test should be used during the requirements phase?
- acceptance
Which test type should be used to compare the program to its initial requirements?
- acceptance
Which test type validates an expected result from an application?
- acceptance
15 categories of Test Cases: Facility
- aka "function" (but not the same as "function testing") - ensure that the functionality in the objectives is implemented - procedure: scan the objectives sentence by sentence, and when a sentence specifies a "what", determine that the program satisfies the "what". - can often times be performed without a computer; a mental comparison of the objectives with the user documentation is sufficient
Principal Processes for achieving objective: Compliance Confirmation
- audit
Which test type should be used before releasing the application for commercial use?
- beta
An informal review of a module of code has been completed. The development team has decided to evaluate the module from an end user perspective. Which method should be used to determine if the code meets quality standards?
- black box
Which test method will verify the functionality of the GUI without knowledge of the underlying code?
- black box
There is a requirement to verify the usability of a product as it is being delivered to ensure overall design and functionality without regard to the internal functioning of the code. Which quality method meets this requirement?
- black box testing
What type of testing activity is "function testing" normally?
- black-box activity
Peer Review Types
- buddy checks - circulation reviews - inspections - walkthroughs - structured walkthroughs
Circulation Review
- can be formal OR informal - useful when reviewers are geographically separated - useful when looking for a large cross section of reviewers - time is not a constraint - objectives: -- improve work -- consider alternatives -- point out problems -- gain consensus from large set of reviewers -- gain input from valuable contributors who cannot be present for a face-to-face review
Testing Principles
- complete testing is not possible - testing is creative and difficult - testing must be planned - testing requires independence - expected results - invalid and unexpected input
Which standards and procedures are used for managing change in an evolving software product?
- configuration management
SDP: External Specifications
- define the exact representation of the program from the end-user point of view - translate the objectives into a precise product specification, viewing the product as a black box and considering only its interfaces and interactions with the end user.
How do we formulate System Test cases?
- design the system test by analyzing the objectives - formulate test cases by analysing the user documentation
Why should we conduct peer reviews?
- detect defects - remove defects - determine product progress - identify potential improvements - uniform and predictable quality of work - cross-training - reduce costs - reduce development time - reduce testing cost and time
15 categories of Test Cases: Usability
- determine how well the end user can interact with the program
SDP: Module Interface Specifications
- develop a precise specification that defines the interface to, and function of, each module
Purpose of Testing
- establish confidence that the program does what it is supposed to do - make lack of quality visible - execute a program with the intent of finding errors - exercise a component to verify that it satisfied a specific requirement - provide continual assessment of whether the software being produced will meet the needs of the user
What is the purpose of a "Module Test"?
- find discrepancies between the program's modules and their interface specifications
Unit Testing
- focuses on testing smaller units of the program first - module specs and source code are required - analyzed using the white-box method - focuses on test statements, branches, and paths through discrete pieces of code
Black Box testing
- functionality testing - does not peer into the internal structures or workings - all levels of testing
Inspector's/Reviewers Responsibilities
- identify and describe the defects found - represent different points of views - be assigned specific review topics to ensure effective coverage
Management has requested that the development team have a moderator review their newest module of code. Which type of review is being asked for?
- inspection
Principal Processes for achieving objective: Verification
- inspection/walkthrough test
Which test should be used during the system design phase?
- integration
Why does the peer rating technique help the programmer develop quality code?
- it allows programmers to self-assess their programming skills
Why can't the Objectives Document be used to formulate test cases?
- it does not contain precise descriptions of the program's external interfaces
Why is integration testing omitted from The Art of Software Testing?
- it is often not regarded as a separate testing step - when incremental module testing is used, it is an implicit part of the module test
How does a test verify that a program meets requirements?
- it validates input conditions that produce defined requirements
Walkthroughs
- less formal verification technique - normally led by the author - objective is to improve work - objective is to consider alternatives - objective is to point out inefficiencies or code problems - exchange techniques and style variations - educate the participants
Regression Testing
- makes sure system modifications will not break the existing features
Condition Coverage
- measures the TRUE and FALSE outcome of each Boolean subexpression - similar to decision coverage, but it has BETTER sensitivity to the control flow
Path Coverage
- measures whether each of the possible paths in each function has been followed - the number of paths is exponential to the number of branches - many paths are impossible to exercise due to relationships of data
Multiple Condition Coverage
- measures whether every possible combination of Boolean subexpression occurs - requires very thorough testing - a disadvantage is that it can e difficult to determine the minimum set of test cases required
Which kind of plan should be in place to proactively face and resolve the risks of the development process?
- mitigation
Which testing process goes along with "SDP: Module Interface Design"?
- module test
How does a software test become complete?
- only by setting out to prove that some form of inputs "work improperly" - assume that the program is "untrue" to its specification and objectives
Which test type determines whether the program meets response and throughput requirements?
- performance
What are the three types of manager reviews?
- project manager reviews - milestone reviews - senior management oversight reviews
A series of defects have recently been found by end users testing new functionality in an existing program, but were fixed by the development team. Which test types should be used to verify the current functionality is working as designed?
- regression
During the implementation phase of a project, the developer realizes the need to change part of the project that has already been written and tested. Which test type should be performed after these changes are made?
- regression
An end user will perform an acceptance test against an application as a final verification. Which development process step is being verified?
- requirements
Which test should be used in the requirements phase?
- requirements
What are the two types of information required for writing a test case?
- specification for the module - module source code
SDP: Program Structure Design
- specify the function of each module, the hierarchical structure of the modules, and the interfaces between modules
Statement Coverage
- statement = logic - measures whether each executable statement is encountered - completely insensitive to the logical operators - cannot distinguish consecutive "switch" labels
15 categories of Test Cases: Stress
- subject the program to abnormally large loads, generally concurrent processing
15 categories of Test Cases: Volume
- subject the program to abnormally large volumes of data to process
Which type of review is used to identify discrepancies from specifications or standards?
- technical
Principal Processes for achieving objective: Validation
- test - simulation - emulation
Integration Testing
- test component interfaces - make sure simple test cases and transactions are being handled properly by the system
What key engineering work products should be maintained under CONFIGURATION MANAGEMENT?
- test plans - test cases - test procedures - test scenarios - test scripts - test data
What is the definition of functional system testing?
- testing the end-to-end functionality of the system as a whole
White Box testing
- tests internal structures or workings of an application - applied at the unit, integration, and system levels - finds unimplemented or missing requirements
Systems Testing
- the entire system is tested against requirements - systems testing measures and determines what the system capabilities are and ends when the system capabilities have been measured and enough of the problems have been corrected to have confidence that the acceptance is ready to be executed
What makes System Testing vitial?
- the number of errors made, and the severity of those errors, are usually greatest in the System Design step
What events must have occurred for an Inspection to occur?
- the product is complete - automated error-detecting tools are available - required supporting documentation is available - for a reinspection, all items noted on the defect list are resolved - the author is available - a trained moderator and sufficient number of skilled inspectors are available
Decision Coverage
- there must be a test case so that every decision has a true and a false outcome, at least once - can usually satisfy statement coverage - measures whether Boolean expressions test in control structures are evaluated to both TRUE and FALSE
SDP: System Design
- this step partitions the system into individual programs, components, or subsystems - defines their interfaces
What is the purpose of a "System Test"?
- to compare the system or program to its original objectives - to show that the product is inconsistent with its original objectives
What is the goal of software testing?
- to find problems
What is the purpose of a "Function Test"?
- to show that a program does not match its external specifications - to expose errors and discrepancies with the specification
Reasons for management reviews?
- track progress - identify inconsistencies - monitor risks - evaluate effectiveness of management - corrective action - project control - change project scope
SDP: Code
- translate, through on or more substeps, the module interface specification into the source code algorithm of each module
What should you focus on when looking for discrepancies between the program and its objectives?
- translation errors made during the process of designing the external specifications
Which test should be used to ensure a module of code meets quality standards?
- unit
Testing Levels
- unit - integration - systems - acceptance - regression
Acceptance Testing
- usability and reliability - confirms if system is ready for operational use - the focus is on functional requirements, performance requirements, operational use profile, and recovery requirements
When is System Testing impossible?
- when there is no set of written, measurable objectives for the product
Which type of test involves coverage of branches, paths, and conditions in the logic of an application?
- white box
What type of testing activity is "module-testing" normally?
- white-box
What are the seven steps of the software development process (SDP)?
1. requirements 2. objectives 3. external specifications 4. system design 5. program structure design 6. module interface specifications 7. code
Which testing process goes along with "SDP: Requirements"?
Acceptance Test
QA proposes a formal group meeting where the programmer narrates, statement by statement, the logic of a program as an error detection technique. Which quality method meets this proposal?
Code inspection
Management has requested that the development team have a moderator review their newest module of code. Which type of review is being requested?
Inspection
Which testing method should be used to compare a developer's program to the program's initial requirements?
system