Software Testing
usability and human computer interaction testing
evaluate how easy it is for end users to learn and use the software
unchanged
for incremental development, the purpose of regression testing is to show that software is ______ by incremental changes to the software, except insofar as it should
prevention
for many organizations, the approach to software quality is _____: it is obviously much better to prevent problems than to correct them
finite state machines
given a set of inputs, you can move to a different state, but you are always in a state which can be tested
defect discovery
in testing for ___ _____, a successful test is one that causes the system to fail
white-box testing
testing that includes an assessment of the software logic and components
fault
the cause of a mlafunction
capture/replay tools
automatically re-execute, or reply, previously executed tests which have recorded inputs and outputs
system testing
concerned with testing the behavior of an entire system considered appropriate for assessing the nonfunctional system requirements (security, speed, accuracy, reliability) external interfaces are evaluated at this level
defect tracking
defects can be tracked or analyzed to determine when they were introduced into the software, why they were created, and when they could have been first observed in the software
defect
can be used to either refer to a fault or a failure, when the distinction is not important
dynamic
testing always implies executing the program on selected inputs
removed
testing can reveal failures, but it is the faults that must be
back-to-back testing
testing in which two or more variants of a program are executed with the same inputs, the outputs are compared, and errors are analyzed in the case of discrepancies
configuration testing
verifies the software under different specified configurations
finite
we generate a _____ set of test cases to test software by selecting a subset of possible tests, which are determined by risk and prioritization criteria
ad hoc testing
tests are derived relying on the software engineer's skill, intuition, and experience with similar program can be useful for identifying test cases that are not easily generated by more formalized techniques
random testing
tests are generated purely at random falls under input domain testing fuzz testing is a special form of random testing
testing objectives
test-related measures that provide an evaluation of the program under test
highlight potential weaknesses
test planning and test designing activities provide useful input for software designers and help to ____ ____ ___, such as design oversights/contradictions, or omissions/ambiguities in the documentation
control flow-based coverage criteria
aimed at covering all the statements, blocks of statements, or specified combinations of statements in a program strongest criteria is path testing
recovery testing
aimed at verifying software restart capabilities after a system crash or other disaster
interface testing
aims at verifying whether the components interface correctly to provide the correct exchange of data and control information
model
an abstract (formal) representation of the software under test or of its software requirements can be used in model-based testing to validate requirements, check consistency and generate test cases
security testing
focused on the verification that the software is protected from external attacks verifies the confidentiality, integrity, and availability of the systems and its data
selection criteria
how we select which test techniques to use can cause major differences in results and degrees of effectiveness
t-wise
in pairwise testing every possible combination of t input variables is considered
usability heuristics
include cognitive walkthroughs, claims analysis, field observations, thinking aloud, and even indirect approaches such as user questionnaires and interviews
beta testing
larger set of representative users test software before it is released
equivalence partitioning
partitioning the input domain into a collection of subsets based on a specific criterion or relation
integration testing
process of verifying the interactions among software components often an ongoing activity at each stage of development with hierarchal structured software incremental integration testing strategies are usually preferred
test harness
provide a controlled environment in which tests can be launched and the test outputs can be logged
test generators
provide assistance in the generation of test cases the generation can be random, path-based, model based or a mix
operational testing
randomly generating test cases according to the operational profile of the software
software testability
refers to the ease with which a given test coverage criterion can be satisfied defined as the likelihood, possibly measured statistically, that a set of test cases will expose a failure if the software is faulty
decision tables
represent the logical relationships between conditions and actions are derived from considering every possible combination of conditions
exploratory testing
simultaneous learning, test design, and test execution tests are not defined in advance in an established test plan, but are dynamically designed, executed, and modified
corrective maintenance
software failures experienced after delivery are addressed by ___ ____
test level
software testing is usually performed at different levels throughout the development and maintenance each level has a target for testing which is called the objective
life cycle
software testing is, or should be, pervasive throughout the entire development and maintenance ___ ____
usability inspection methods
specialized heuristics applied for the systemic observation of system usage under controlled conditions in order to determine how well people can use the system and its interfaces
workflow models
specify a sequence of activities performed by humans and/or software applications where each sequence of actions constitutes one workflow (aka scenario)
accomplished
test adequacy criteria can be used to decide when sufficient testing will be, or has been ____
boundary value analysis
test cases are chosen on or near the boundaries of the input domain of variables
pairwise testing
test cases are derived by combining interesting values for every pair of a set of input variables instead of considering all possible combinations
deterministic testing
test cases are selected for most important requirements to make sure those are functioning in the software
error guessing
test cases are specifically designed by software engineers who try to anticipate the most plausible faults in a given program
data flow-based criteria
the control flow graph is annotated with information about how the program variables are defined, used, and killed
test result evaluation
the results of testing should be evaluated to determine whether or not the testing has been successful successful means software performed as expected and did not have major unexpected outcomes
regression testing
the selective retesting of a system or component to verify that modifications have not caused unintended effects and that the system or component still complies with its specified requirements
performance testing
verifies that the software meets the specified performance requirements and assess performance characteristics (instance, capacity, and response time)
unit testing
verifies the functioning in isolation of software elements that are separately testable occurs with access to the code being tested and with the support of debugging tools
fault, failure
a _____ creates a _____
effectiveness
testing ____ is determined by analyzing a set of program executions
testing log
testing activities can be entered to identify when a test was conducted, who performed the test, what software configuration was used, and other relevant identification information
tradeoff
testing always implies a _____ between limited resources and schedules on one hand and inherently unlimited test requirements on the other
requirements
testing may not run the program but it can only be considered working correctly if it meets all the ______
black-box testing
testing that doesn't reference or know what the internal software does, only cares about inputs or outputs
sufficient for a specified purpose
a test selection criteria is means of selecting test cases or determining that a set of test cases
testing techniques
aids that help to ensure achievement of test objectives the measures that evaluate the thoroughness of the test set ex: branching
failure
an undesired effect observed in the system's delivered service
oracle/file comparators/assertion checking tools
assist in deciding whether a test outcome is successful or not
test-related measures
the factors you are hoping to maximize while testing (speed, efficiency, reliability, security, etc.) test important modules that are used frequently
fault seeding
faults are placed artificially into a program before testing when the test are executed, some of these fault will be revealed along with other faults that could not have been seen before
robustness testing
extension of boundary-value analysis test cases are also chosen outside the input domain of variables to test program robustness in processing unexpected or erroneous inputs
program testing
___ ____ can be used to show the presence of bugs, but never to show their absence (Dijkstra's)
terminated
a decision must be made as to how much testing is enough and when a test stage can be ______
mutation testing
a mutant is a slightly modified version of the program under test, differing from it by a small syntactic change uses the coupling effect
alpha testing
a small selected group of potential users test the program in trial use before release of software
life test, reliability evaluation
a statistical estimate of software reliability, which can be obtained by observing reliability achieved, can be used to evaluate a software product and decide whether or not testing can be stopped
usability principles
can provide guidelines for discovering problems in the design of the user interface
testing for verification
checking observed behavior of a program and checking against a specification
testing for validation
checking observed behavior of a program and checking against user needs
software testing
consists of dynamic verification that a program provides expected behaviors on a finite set of test cases, suitably selected (most important)
infeasible path
control flow paths that cannot be exercised by any input data testing every path would be impossible, could be infinite
coverage analyzers and instrumenters
coverage analyzers assess which and how may entities of the program flow graph have been exercised amongst all those required by the selected test coverage criterion
acceptance testing
determines whether a system satisfies its acceptance criteria, usually by checking desired system behaviors against the customer's requirements
stress testing
exercises software at the maximum design load with the goal of determining the behavioral limits