Software Testing

usability and human computer interaction testing

evaluate how easy it is for end users to learn and use the software

unchanged

for incremental development, the purpose of regression testing is to show that software is ______ by incremental changes to the software, except insofar as it should

prevention

for many organizations, the approach to software quality is _____: it is obviously much better to prevent problems than to correct them

finite state machines

given a set of inputs, you can move to a different state, but you are always in a state which can be tested

defect discovery

in testing for ___ _____, a successful test is one that causes the system to fail

white-box testing

testing that includes an assessment of the software logic and components

fault

the cause of a mlafunction

capture/replay tools

automatically re-execute, or reply, previously executed tests which have recorded inputs and outputs

system testing

concerned with testing the behavior of an entire system considered appropriate for assessing the nonfunctional system requirements (security, speed, accuracy, reliability) external interfaces are evaluated at this level

defect tracking

defects can be tracked or analyzed to determine when they were introduced into the software, why they were created, and when they could have been first observed in the software

defect

can be used to either refer to a fault or a failure, when the distinction is not important

dynamic

testing always implies executing the program on selected inputs

removed

testing can reveal failures, but it is the faults that must be

back-to-back testing

testing in which two or more variants of a program are executed with the same inputs, the outputs are compared, and errors are analyzed in the case of discrepancies

configuration testing

verifies the software under different specified configurations

finite

we generate a _____ set of test cases to test software by selecting a subset of possible tests, which are determined by risk and prioritization criteria

ad hoc testing

tests are derived relying on the software engineer's skill, intuition, and experience with similar program can be useful for identifying test cases that are not easily generated by more formalized techniques

random testing

tests are generated purely at random falls under input domain testing fuzz testing is a special form of random testing

testing objectives

test-related measures that provide an evaluation of the program under test

highlight potential weaknesses

test planning and test designing activities provide useful input for software designers and help to ____ ____ ___, such as design oversights/contradictions, or omissions/ambiguities in the documentation

control flow-based coverage criteria

aimed at covering all the statements, blocks of statements, or specified combinations of statements in a program strongest criteria is path testing

recovery testing

aimed at verifying software restart capabilities after a system crash or other disaster

interface testing

aims at verifying whether the components interface correctly to provide the correct exchange of data and control information

model

an abstract (formal) representation of the software under test or of its software requirements can be used in model-based testing to validate requirements, check consistency and generate test cases

security testing

focused on the verification that the software is protected from external attacks verifies the confidentiality, integrity, and availability of the systems and its data

selection criteria

how we select which test techniques to use can cause major differences in results and degrees of effectiveness

t-wise

in pairwise testing every possible combination of t input variables is considered

usability heuristics

include cognitive walkthroughs, claims analysis, field observations, thinking aloud, and even indirect approaches such as user questionnaires and interviews

beta testing

larger set of representative users test software before it is released

equivalence partitioning

partitioning the input domain into a collection of subsets based on a specific criterion or relation

integration testing

process of verifying the interactions among software components often an ongoing activity at each stage of development with hierarchal structured software incremental integration testing strategies are usually preferred

test harness

provide a controlled environment in which tests can be launched and the test outputs can be logged

test generators

provide assistance in the generation of test cases the generation can be random, path-based, model based or a mix

operational testing

randomly generating test cases according to the operational profile of the software

software testability

refers to the ease with which a given test coverage criterion can be satisfied defined as the likelihood, possibly measured statistically, that a set of test cases will expose a failure if the software is faulty

decision tables

represent the logical relationships between conditions and actions are derived from considering every possible combination of conditions

exploratory testing

simultaneous learning, test design, and test execution tests are not defined in advance in an established test plan, but are dynamically designed, executed, and modified

corrective maintenance

software failures experienced after delivery are addressed by ___ ____

test level

software testing is usually performed at different levels throughout the development and maintenance each level has a target for testing which is called the objective

life cycle

software testing is, or should be, pervasive throughout the entire development and maintenance ___ ____

usability inspection methods

specialized heuristics applied for the systemic observation of system usage under controlled conditions in order to determine how well people can use the system and its interfaces

workflow models

specify a sequence of activities performed by humans and/or software applications where each sequence of actions constitutes one workflow (aka scenario)

accomplished

test adequacy criteria can be used to decide when sufficient testing will be, or has been ____

boundary value analysis

test cases are chosen on or near the boundaries of the input domain of variables

pairwise testing

test cases are derived by combining interesting values for every pair of a set of input variables instead of considering all possible combinations

deterministic testing

test cases are selected for most important requirements to make sure those are functioning in the software

error guessing

test cases are specifically designed by software engineers who try to anticipate the most plausible faults in a given program

data flow-based criteria

the control flow graph is annotated with information about how the program variables are defined, used, and killed

test result evaluation

the results of testing should be evaluated to determine whether or not the testing has been successful successful means software performed as expected and did not have major unexpected outcomes

regression testing

the selective retesting of a system or component to verify that modifications have not caused unintended effects and that the system or component still complies with its specified requirements

performance testing

verifies that the software meets the specified performance requirements and assess performance characteristics (instance, capacity, and response time)

unit testing

verifies the functioning in isolation of software elements that are separately testable occurs with access to the code being tested and with the support of debugging tools

fault, failure

a _____ creates a _____

effectiveness

testing ____ is determined by analyzing a set of program executions

testing log

testing activities can be entered to identify when a test was conducted, who performed the test, what software configuration was used, and other relevant identification information

tradeoff

testing always implies a _____ between limited resources and schedules on one hand and inherently unlimited test requirements on the other

requirements

testing may not run the program but it can only be considered working correctly if it meets all the ______

black-box testing

testing that doesn't reference or know what the internal software does, only cares about inputs or outputs

sufficient for a specified purpose

a test selection criteria is means of selecting test cases or determining that a set of test cases

testing techniques

aids that help to ensure achievement of test objectives the measures that evaluate the thoroughness of the test set ex: branching

failure

an undesired effect observed in the system's delivered service

oracle/file comparators/assertion checking tools

assist in deciding whether a test outcome is successful or not

test-related measures

the factors you are hoping to maximize while testing (speed, efficiency, reliability, security, etc.) test important modules that are used frequently

fault seeding

faults are placed artificially into a program before testing when the test are executed, some of these fault will be revealed along with other faults that could not have been seen before

robustness testing

extension of boundary-value analysis test cases are also chosen outside the input domain of variables to test program robustness in processing unexpected or erroneous inputs

program testing

___ ____ can be used to show the presence of bugs, but never to show their absence (Dijkstra's)

terminated

a decision must be made as to how much testing is enough and when a test stage can be ______

mutation testing

a mutant is a slightly modified version of the program under test, differing from it by a small syntactic change uses the coupling effect

alpha testing

a small selected group of potential users test the program in trial use before release of software

life test, reliability evaluation

a statistical estimate of software reliability, which can be obtained by observing reliability achieved, can be used to evaluate a software product and decide whether or not testing can be stopped

usability principles

can provide guidelines for discovering problems in the design of the user interface

testing for verification

checking observed behavior of a program and checking against a specification

testing for validation

checking observed behavior of a program and checking against user needs

software testing

consists of dynamic verification that a program provides expected behaviors on a finite set of test cases, suitably selected (most important)

infeasible path

control flow paths that cannot be exercised by any input data testing every path would be impossible, could be infinite

coverage analyzers and instrumenters

coverage analyzers assess which and how may entities of the program flow graph have been exercised amongst all those required by the selected test coverage criterion

acceptance testing

determines whether a system satisfies its acceptance criteria, usually by checking desired system behaviors against the customer's requirements

stress testing

exercises software at the maximum design load with the goal of determining the behavioral limits