Introduction to Software Engineering Test 3
What are the 4 steps of testing?
- Unit testing - Integration testing - High-order testing group (steps 3 and 4) -Validation testing -Systems testing
What are the benefits of smoke testing?
-Integration risk is minimized. -The quality of the end product is improved. -Error diagnosis and correction are simplified. -Progress is easier to assess.
What are the 5 focuses of unit testing?
-Module interface -Local data structures -Boundary conditions -All independent paths -Error-handling paths
What are the roles of the SQA group?
-Prepares an SQA plan for a project. -Participates in the development of the project's software process description. -Reviews software engineering activities to verify compliance with the defined software process. -Audits designated software work products to verify compliance with those defined software process. -Ensures that deviations in software work and work products are documented and handled according to a documented procedure. -Records any noncompliance and reports to senior management.
What is the test specification comprised of?
-Test plan -Test procedure -Test Report
What are the different strategies of integrating for OO testing?
-Thread based testing -Use based testing -Cluster testing
Bottom Up Integration
-drivers are replaced one at a time, "depth first" -worker modules are grouped into builds and integrated.
Top Down Integration
-top module is tested with stubs -stubs are replaced one at a time, "depth first" -as new modules are integrated, some subset of tests is re-run
Validation
A different set of tasks that ensure that the software has been built is traceable to customer requirements; Are we building the right product?
Six-Sigma for Software Engineering
Define Measure Analyze Improve Control
How does effort differ with reviews?
Effort expended with reviews will be slightly increased during the design and coding process, however, the reduced amount of effort and time needed to test and deploy the product is significant enough to say that the reviews are well worth the effort.
Systems testing
Elements mesh and achieves overall system function/performance; The second step of high-order testing and the fourth step of the testing process
Effective Software Process
Establishes the infrastructure that supports any effort at building a high-quality software product.
Design quality
Every element of the design model should be assessed by the software team to ensure that it exhibits high quality and that the design itself conforms to requirements.
Scheduling Decisions
Failing to pay attention to task dependencies when creating the project schedule
For Web Applications
From TITS to TITL: Content -> Interface -> Navigation -> Functional component -> Compatibility -> Performance -> Stress
Quality Factors
Functionality Reliability Usability Efficiency Maintainability Portability
Three Types of Costs in Quality
Prevention costs, Internal failure costs, and External failure costs
Who all is involved in a review meeting?
Producer, review leader, reviewer(s), and recorder
Project Management
Project plan includes explicit techniques quality and change management
Two types of software quality
Quality of design and quality of conformance
Quality control
Series of inspections, reviews, and tests used to ensure conformance of a work product to its specifications
What causes software quality?
Software quality is the result of good project management and solid engineering practice
Code quality
Source code and related work products must conform to local coding standards and exhibit characteristics that will facilitate maintainability.
Elements of SQA
Standards Reviews and Audits Testing Error/defect collection and analysis Change management Education Vendor management Security management Safety Risk Management
Reliability
The amount of time that the software is available for use as indicated by the following sub-attributes: maturity, fault, tolerance, and recoverability.
Informal Review
This is typically done with either a deck check of a software engineering work with a colleague, a casual meeting for the purpose of reviewing a work product, or the review-oriented aspects of pair programming.
Software Engineering MEthods
To build high quality software, you must understand the problem to be solved and be capable of creating a quality design that conforms to the problem requirements Eliminating architectural flaws during design can improve quality
What are the objectives of a Formal Technical Reviews?
To uncover errors in function, logic, or implementation for any representation of the software To verify that the software under review meets its requirements To ensure that the software has been represented according to predefined standards To achieve software that is developed in a uniform manner To make projects more manageable
Quality of conformance
an issue focused primarily on implementation
What is software quality assurance?
an umbrella activity that is applied throughout the software process. it is also called quality management
User satisfaction
compliant product, good quality, and delivery within the budget and schedule
External failure costs
compliant resolution, product return and replacement, help line support, and warranty work
Quality assurance
consists of the auditing and reporting procedures used to provide management with data needed to make proactive decisions
Local data structures
data maintains integrity during all of execution
"Good Enough" Software
delivers high quality functions and features that end-users desire, but at the same time it delivers other more obscure or specialized functions and features that contain known bugs.
Useful Product
delivers the content, functions, and features that the end-user desires
Three important points of software quality
effective software process useful product adds value for the producer and the user
Quality of design
encompasses requirements, specifications, and the design of the system
Types of management decisions
estimation decisions, scheduling decisions, and risk-oriented decisions
All independent paths
execute all paths at least once
Adding Value
high quality software provides benefits for the software organization and the end-user community
Module interface
information flow in and out of unit
Boundary conditions
module operates properly at boundaries
Prevention costs
quality planning, formal technical reviews, test equipment, and training
Regression testing
re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects
Risk-oriented Decisions
reacting to each crisis as it arises rather than building in mechanisms to monitor risks may result in products having reduced quality
Internal failure costs
rework, repair, and failure mode analysis
Error-handling paths
tested and correct
The FTR is actually a class of reviews that includes what?
walkthroughs and inspections
What are reviews?
A meeting conducted by technical people for technical people A technical assessment of a work product during the software engineering process A software quality assurance mechanism A training ground
What are not examples of reviews?
A project summary or progress assessment A meeting intended solely to impart information A mechanism for political or personal reprisal
Software safety
A software quality assurance activity that focuses on the identification and assessment of potential hazards that may affect software negatively and cause an entire system to fail.
Quality control effectiveness
A software team should apply limited resources in a way that has the highest likelihood of achieving a high quality result.
The Review Meeting will result in one of three things:
Acceptance of the product Reject the product due to severe errors Accept the product provisionally
Test Report
Actual test results, problems, and peculiarities are recorded
What does software quality assurance encompass?
An SQA process Specific quality assurance and quality control tasks Effective software engineering practice Control of all software work products and the changes made to them A procedure to ensure compliance with software development standards Measurement and reporting mechanisms
Software Quality
An effective software process applied in a manner that creates a useful product that provides measurable value for those who produce it and those who use it
What is the difference between an error and a defect?
An error is a quality problem found before the software is released to end users, but a defect is a quality problem found only after the software has been released to end-users.
Smoke Testing
An integration testing approach that is commonly used when product software is developed. It is designed as a pacing mechanism for time-critical projects, allowing the software team to asses the project on a frequent basis
What should you do to perform effective testing?
Conduct effective technical reviews. This will lead to many errors being eliminated before testing commences.
Who is the recorder?
The reviewer who records all important issues raised during the review in writing.
Verification
The set of tasks that ensure that software correctly implements a specific function; Are we building the product right?
Smoke Testing Steps
1.) Software components that have been translated into code are integrated into a "build". 2.) A series of tests is designed to expose errors that will keep the build from properly performing its function. 3.) The build is integrated with other builds, and the entire project is smoke tested daily.
What is the best way to do integration testing?
Incremental construction
General Testing Criteria
Interface integrity Functional validity Information content Performance
Interface integrity
Internal and external interfaces are tested as each module is integrated
Estimation Decision
Irrational delivery date estimates cause teams to take short-cuts that can lead to reduced product quality
How does pair programming benefit quality?
It encourages continuous review as a work product is created and provides the benefit of finding many errors immediately.
Mean Time Between Failures (MTBF)
MTBF = MTTF + MTTR (MTTF - Mean time to failure, MTTR - Mean time to repair)
Validation testing
Meets functional, behavior, and performance requirements; The first step of high-order testing and the third step of the testing process
Test procedure
Order of integration and corresponding test at each integration step is described
The Software Quality Dilemma
People in the industry try to get a magic middle ground where a product is good enough to not be immediately rejected, but also not perfect the object so that it doesn't take too long. This results in good enough software.
What are the 4 SQA goals?
Requirements quality Design quality Code quality Quality control effectiveness
For conventional software
TITS: The module (component) TITL: Integration of modules
For OO software
TITS: an OO class that encompasses attributes and operations TITL: The communication and collaboration of said OO class
Information content
Test for errors in local or global data structures
Incremental Construction
Test program in small increments which makes errors easier to isolate and correct; Examples include top-down integration, bottom-up integration, and sandwich testing
Functional validity
Test to uncover functional defects in the software
How does testing start?
Testing begins at the component level and works "outward" toward the integration of the entire computer-based system
Test Plan
Testing divided into phases and builds that address specific functionality and behavioral characteristics
Unit testing
Tests component control structures and is the first step of the testing process.
Requirements quality
The correctness, completeness, and consistency of the requirements model will have a strong influence on the quality of all work products that follow.
Usability
The degree to which the software is easy to use as indicated by the following sub-attributes: understandability, learnability, and operability.
Efficiency
The degree to which the software makes optimal use of system resources as indicated by the following sub-attributes: time behavior, resource behavior.
Functionality
The degree to which the software satisfies stated needs as indicated by the following sub-attributes: suitability, accuracy, interoperability, compliance, and security
Maintainability
The ease with which repair may be made to the software based on analyzability, changeability, stability, and testability.
Portability
The ease with which the software can be transposed from one environment to another based on adaptability, installability, conformance, and replaceability.
How do the different types of cost of quality vary in cost.
The relative costs to find and repair an error or defect increase dramatically as we go from prevention to detection to internal failure to external failure costs.
Who is the producer?
The individual who has developed the work product. Also informs the project leader that the work product is finished and requires review.
Who is/are the reviewer(s)?
The people who are expected to spend between one and two hours reviewing the product, making notes, and otherwise becoming familiar with the work.
Who is the review leader?
The person who evaluates the product for readiness, generates copies of product materials, and distributes them to two or three reviewers for advance preparation.
Software availability
The probability that a program is operating according to requirements at a given point in time
Software testing
The process of exercising a program with the specific intent of finding errors prior to the delivery to the end user
Defect Amplification Model
Used to illustrate the generation and detection of error during the design and code generation actions of a software process.
Integration testing
Verification, program construction, and the second step of the testing process
Performance
Verify specified performance bounds are tested
How do we test?
We begin by 'testing-in-the-small' (TITS) and move toward 'testing-in-the-large' (TITL)
Availability
[MTTF/(MTTF+MTTR)]X100%
Quality (dictionary definition)
a characteristic or attribute of something
