Software quality assurance 2

pdf - probability density function

# defects which will be found per life cycle phase

test effectiveness

(# defects removed by phase / # total # defects found in this hase or later phase)

Calculating DRE v2 - steps

1) take req% and place in ID box for D_req. ID * (1 - % of FE (given)) = PD. ID - PD = RD sum to get totals per row

3 critera to follow

1) use every def, 2) get to every use 3) follow all du paths

prime path

: A simple path that does not appear as a proper subpath of any other simple path

perfective maintenance

: Modification of a software product after delivery to improve performance or maintainability

phase containment effectiveness

= (# phase i errors) / (number of phase i errors + number of phase i defects)

simple path

A path from node ni to nj is simple if no node appears more than once, except possibly the first and last nodes are the same. no internal loops

preventative maintenance

Modification of a software product after delivery to detect and correct latent faults in the software product before they become effective faults.

def-clear

a path from i to j is def clear for var v if v is not given another value on any of the nodes or edges in the path

partitioning

a sampling strategy to simplify testing. subjective due to ec picking process

basic block

a sequence of statements that if the fisrt statement is executed, all statements will be (no brances)

du-path

a simple subpath that is def=clear with respect to v from a def of v to a use of v

tour

a test pat p tours subpath q if q is a subpath of p. side trips (edge in same order)and detours (node in same order)

galin extended cost model

adds managerial costs to each to prepare and to handle failure costs

sci verison software configuration item version

approved state of sci at any given time

sci - software configuration item

approved unit of software code for config mgmt and treated as distinct entity

Calculating DRE V1 - steps

assume 100% defects, get average % of defects originating in each phase, calculate cost of each plan via matrix

data flow coverage

augmenting the control flow graph

equivalence class partintioning

blackbox aimed to increase efficiency of testing and improve coverage

peer reviews

both inspections and code walkthrough (formal and less formal) review leader, author of review, and specialized professions. Feedback only, not formal sign off

limitation of metrics

budgetary constraints allocating resources, human factors, biased reporting, partial reporting

tools for maintance

bug repo, debugging tools, versioning tools, configuration mgmt, testing tools

timetable metrics

calculate milestones completed on time / total # of milestones

auto vs manual testing

can automate performing tests, prepping test log, regression tests. manual planning, design, and test setup

fuzzy boundaries

can't use mutual exclusive ranges since too many combinations. must use EC instead

tasks of config mgmt

control software change (proposed changes), release sci and versions, verify compliace with scm procedures. consider urgency of change, effect it will have on timetables and testing, cost of change

components of maintenance

corrective maintenances (bug fix), adaptive maintanance (changes for new os, etc), and functionality improvement maintenance (perfective and preventative maintenanace)

2 types of costs

cost of control (prevention and appriasal) cost of failure of control (internal failure costs, extrenal failure costs)

RDRC

cost of defect removal per phase. unit cost of removal

external failure costs

customer complaints, damages, reduction of sales to customers, increase in sales team cost to raise sale amount

structural coverage criteria

defined on a graph in terms of edges and nodes

test requirements

describe properties of test paths

software plan template

determine scope of tests, testing environemnt, test details, test schedule. test results, summary for totoal # errors, special events and proposals

testing process (4 steps)

determine strategy, plan tests, design tests, peform tests

all-du-path coverage

each and every paths between all defs and uses

availability

full availability, vital available, total unavailability (down time failure hours / hours software in service)

why config mgmt?

handling correct version for next development task, who can provide what tool or document, what changes have been made since A happened, where can i find who uses version 1.1 of our software

defect injection

if defects carry over from previous steps, they are injected into the next phase dre = defects removed at step / (defects existing + defects injected)

prevention cost

investingg in sqa prevention activities, infrastructure

kloc

kilolines of code. determine quality by lines of code

software evolution models

linear and tree branching and successive changes for releases

maintanance plan

list of contractual servies agreed upon. scope budget, schedule personnel

software quality metrics requreiements

must be relevant, valid, reliable, comprehensible, mutually exclusive

error density metrics

normalizes by dividing by kloc or wce or nfp (number of coding errors/kloc)

nce

number of coding errors

nfp

number of cuntion points - function point is a "unit of measurement" to express the amount of business functionality an information system (as a product) provides to a user. Function points are used to compute a functional size measurement (FSM) of software. The cost (in dollars or hours) of a single unit is calculated from past projects

DRE metrics

number of development errors / (number of errors + number of failures a year)

motorola

number of prelease defects / (number of prerelease defects + number of post release defects)

reliability

probabilyt of failure free operation over specified time

software quality metics

quantitative measure of degree that an item is of quality

test case sources

random sampling of real life cases and synthetic test cases (simulated)

metrics of reliability

rate of fault occurrence, prob of failure on demaind, MTTF, # transactions, # calendar dates, raw execution time

dynamic reliability modesl

rayleigh model, exponential model

internal failure cost

redesign due to test findings, repeating design reviews due to test findings

appriasal cost

reviwing, testing, getting external participants

version development policy

sequential version policy tree version policy

process metrics categories

software quality metrics, software proces time table metrics, DRE metrics, productivity metrics

software config mgmt

sqa component for applying tools and procedures that allow completion of tasks required to maintain sci and software configuration versions

baseline version

stable and relsead version that was tested thoroughly

types of reliability modesl

static (uses product characteristics to estimate # defects) and dynamic (uses statistical distribution to estimate future reliability)

3 types of external participants

subcontractors, 3rd party provider of modules, customers

path(t)

test path is executed by test t

cdf - cumulative density function

total # defects will be found as a function of life cycle phase

domain testing

two paths are equivalent if the program would take same path in response to each

equivalence

two test cases are equivalent if you expect the same result for each

testing process and risk

use risk to prioritize test cases ranking. quality plans need to include test plans (shrine)

helpdesk quality metrics

uses hd calls to calculate. # calls/ year. corrective code required in maintenance, # fp to be maintained

productivity metrics

uses working hours. poor metric

du paths

using du pairs, get paths that are unique

release docs

version #, revision #, date, list of installations, list of sci in version, installation instructions, changes in new version, further development process

raylegh model

weibull model with a m value of 2. formal paramtric model to produce estimate of future defect rate

wce

weight of coding errors (coding errors x relative weight)

error severity metrics

weighted code errors / number of code errors to get average severity of code errors (uses weights provided to compare on severity)

defect origin

where defects are first introduced

equivalence class

- set of input variable values that produce the same output results or are processed identically.

bottom up testing

REquires Test Driver. test all lower levels, the combine and test one level higher

top down testing

Requires Stubs. test higher functionality with stubs at the lowest level. then implement downwards.

Phases of software development RDU ID SO

Requirmenents, design, unit coding, integration, documentation, system testing,operation

types of testing

alpha (end users at dev's site), beta (end users in their environment) , uat (testing for sign off), performance testing (load and stress), security testing, UX testing, unit, integration, subsystem, system testing

prime path coverage

determine all simple paths (no duplicate nodes except first and last), then remove all duplicates or subpaths

What is the inverted v diagram

dev activity - requirements, system/HLD, architecture design, module design Coding unit testing, integration testing, system testing, UAT

boehm and ross's top 10 software risk items

developing wrong functions, unrealistic schedule and budget, wrong interface for user, gold plating, continuous stream of req changes, personnel shortfalls

validation process

does product satisfy specified requirements

verification process

does the product satisfy conditions at the start of the phase

all use coverage

every def reaches all possible uses

edge coverage

execute every branch

node coverage

execute every statement

FE

filtering effectiveness

pairwise testing

for each variable it is tested with every other value

software testing

formal process carried out by testing team, run the programs on computer. follows test procedures or test cases

test path

from initial node to final node

satisfaction

given a set TR of test requriments for criteria C, a set of tests T satisfies C on a graph iif for every test requirement in TR there is a test path in path T that meets the test requiremnt tr

Defect Removal Efficiency

gives a measure of the development team ability to remove defects prior to release. It is calculated as a ratio of defects resolved to total number of defects found. It is typically measured prior and at the moment of release.

def-use important!

id def and use appear on same node, then it is only a du if the def occurs after the use and the node is a loop

loops in graphs

if loop, complete path coverage not feasible, infinite # of paths

reach

if there is def clear path from i to j with var v then i reaches the use of j

Black box testing

ignores internal mechanisisms of system or component and focuses on outputs in response to give inputs. tests compliance with specified requirements

integration testing stratigies

incremental testing (bottom up, top down)or big ban testing (test al at once)

qualification process

is system or component suitible for operational use

formal design reviews

leader, review team of project memebers, customers, professionals outside project. YOu don't fix in design reviews. you sign off formally, approve document, verify and validate comments, decide action items. approve, partial approve, or denial of design review at end. DR report is summary of outcome

element of plan for software quality

list of quality goals, review activities, software tests, accepptance tests for external software, procedures and dates for versions

boundaries

mark the point or zone of transition from one EC to another

model's quantitiative result for DRE

measure total effectiveness at removing defects, and the total costs to remove defects

invalid EC

negative testing classes where you select only one value that is incorrect to make sure error is handled correctly

if statement

one node, two edges (for each condition), two nodes for block in each if statment, and then one node for end of if to return back to regular flow (diamond shaped)

du pairs

pairs of nodes from def to use

PD

passed defects

def-use

path from def to use

POD

phase originated defect

round trips

prime path that starts and ends at same node

factors that affect required intensity of sqa activities

project factors (technical complexity and difficulty, severity of failure outcomes, project magnitude), team factors (professional qualification of team members, availability, experience, familiarity (new team members)),

Objectives of having a development and quality plan

recruit team members, decide if waterfall or agile, resolve dev risks, milestones, sprint lengths, give mgmt data for project control

RD

removed defects

data flow coverage criteria

requires a graph to be annotated with references to variables

edge-pair coverage

requires edges or subpaths of length 2

loops in cfg

requires extra nodes added. while condition is met, arrow points back to initial node at end of block execution

test criteria

reules that define test requirements

best effort touring -

satisfy as many TR as possible. allow sidetrips to satisfy unsatisfiable test requirements

Classes of software dev risks

scheduling/timing risks, system functionality risks, subcontracting risks, requirement mgmt risk, resource usage and performance risk, personnel mgmt risk. must prioritize risks!

equivalence class boundaries

single numeric, alphabet value, group of alphanumerics, range of values, etc

Elements of plan for devs

specify deliverables, define project interfaces, methodolity and tools, milestones, map of dev process, cost estimate

nodes in cfg

statements or sequences of statements (basic blocks)

def

statements that assign values to variables

use

statements that use variables

2 testing harness/scaffold.

stubs and drivers

path(T)

test path is excuted by set of tests T

du-tours

test path tours subpath and the subpath is defclear with respect to v

Edge Coverage tR and Test path

test requirement - must contain every single edge at least once. must contain subpaths of length 1

node coverage

test requirement {123456} must contain every node once

node coverage

test set T satisfies node coerage iif every reachable node there is some path that visits the node

White box testing

testing that takes into account the internal mechanism of a system or component

return statements inside if block in cfg

the two choices will not converge and the return statment will be the finite state (double thick line)

combinatorial testing

to overcome short come of minimum test cases, can test every combination (while trying to minimize # test cases.)

TRC

total removal cost

complete path coverage

tr contains all paths in graph

complete round trip coverage

tr contains all round trop paths for each reachable node in g

all def coverage

tr contains at least one path from du-paths set. every def reaches a use

simple round trip coverage

tr contains at least one rount trip path for each reachable node that begins and ends in round trip path

edge coverage

tr contains each reachable path of length up to 1 in graph (single edge)/ special case of graph of a single node (a graph with one node will not have any edges)

edge pair coverage

tr contains each reachable path of length up to 2

specified graph coverage

tr contains set of test paths where s is a parameter

edges

transfers of control

infeasible test requirements

unreachable statement, subpath that requires contradiction (both > and < 0)

loops

whiles, for etc

line coverage

white box testing concept - percent of program code lines included in planned testing

Path coverage

whitebox testing concept. path coverage is measured by percentage of all possible program plaths included in planned testings