5) Measurement

Mary Shaw

"Lines of code is a measure of input, not output."

Galileo Galilei

"Measure what can be measured, and make measurable what cannot be measured."

Bill Gates

"Measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs."

Tom Gilb, 1988

"Projects without clear goals will not achieve their goals clearly."

Goodhart's Law

"When a measure becomes a target, it ceases to be a useful measure.

Tom Demarco, 1982

"You cannot control what you cannot measure."

Cyclomatic Complexity Formula

=E - N + 2P E = Number of edges. N = Number of nodes. P = Number of procedures. E = 12. N = 9. P = 1 CC = 12 - 9 + 2 = 5

Representational Theory of Measurement

A measurement is a function: M:£-R. The measurement of some entity A is denoted: M(A). M = The measurement function. £ = Observable entities (e.g buildings) R = A number (e.g age).

Meaningful Statements

A statement about a system is meaningful if it remains true for any admissible transformation of the measures involved. - Tower X is two thirds the height of tower Y. The scale is ordinal.

Scale - Interval

Allows for a degree of difference between elements, but not the ratio. e.g angle, celsius. Difference between measures becomes significant, but the exact magnitude of the numbers does not. - Makes no sense to say that difference from 20 degrees to 80 degrees is more than twice the difference from 30 degrees to 60 degrees. Makes no sense to say that 100 degrees is twice as hot as 50 degrees. Translate to fahrenheit and see why.

GQM - Quantitative Level

Associated with every question is a set of data, either subjective or objective, that helps provide a quantitative answer.

Representation Condition

Behaviour of measures in a number system needs to reflect difference between corresponding elements. e.g For every paid of books, the book that is of a higher quality must have higher sales figures. Formally: A measurement M must map entities to numbers and empirical relations to numerical relations such that: - Empirical relations preserver numerical realtions. - Empirical relations are preserved by numerical relations.

GQM Levels

Conceptual Level. Operational Level. Quantitative Level.

Programmer Productivity

Contentious subject. Some organisations reward developers based on such measures. Must be treated with care. Productivity = Lines of code / Hours.

Necessity for Process Improvement

Data from past projects serves as a reference point against which we can measure our effectiveness. Essential to show that changes are cost-effective.

Cyclomatic Complexity

Devised by McCabe. Attempt to count the number of paths through a function. Most paths means that the function is more complex. Harder to test and understand. Defined in terms of the Control Flow Graph.

Scale - Nominal

Different elements have different symbols - no order or distance. e.g types of music: classical, pop, jazz... A classification scheme. e.g Classifying quality properties of a software system: - Efficiency, usability, portability, correctness, accuracy. No notion of ordering between elements. Any one-to-one mapping is an admissible transformation. Only statements concerning equality and inequality and meaningful.

System Spoilage

Effort that is required to fix faults in relation to the total development effort. Gives an insight into the cost of quality. Can be used to justify quality assurance techniques, and measure their success. Spoilage = effort fixing faults / total development effort.

Scale - Ordinal

Elements are assigned a rank, but no notion of distance between adjacent ranks. e.g likest scales - strongly disagree, disagree, agree, strongly agree. An ordinal scale allows attributes to be ranked. e.g How often does your software fail? Never or extremely often on scale from 0 to 4. Only meaningful statements are concerning equality and order.

Cohesion

Extent of activity within a module. Can also be measured in terms of calls, data accesses.

Coupling

Extent of dependencies between two modules. e.g Calls between methods or data accesses.

GQM

Goal Question Metric

Defect Containment

Goal: Establish proportion of faults detected at one phase before percolating to later phases. Question: For each stage, how many faults that originated there are discovered at each stage? Measure: Bug reports tagged as "requirements and "source".

Representative

Health of a person - Body Mass Index. Quality of an essay - Number of words. Success of a government - Increase in GDP. Success of an academic - Number of papers published. Quality of a book/music album - Number of sales.

GQM - Conceptual Level

Measurement goals involve products, processes or resources.

Process Metrics

Metrics are vital to ensure the effectiveness of a software development process. A complex activity. - Lots of goal, many more questions, and a huge number of possible metrics.

Scales

Nominal. Ordinal. Interval. Ratio.

How complex is the code?

Possible the most popular goal metrics. Can determine, for example, amount of effort to test it, need to re-engineer it, cost. Most popular metric is McCabe's Cyclomatic Complexity. To look at the source code logic structure, and count the possible paths through it.

Goal Question Metric Approach

Proposed by Victor Basili in 1994. Adopt a top-down approach to metrics. 1. Start off from a set of specific measurement goals. 2. For each goal, determine the list of questions that must be answered to determine that it is being met. 3. Decide what needs to be measured to answer there questions.

GQM - Operational Level

Questions try to characterise object of measurement in context of a specific quality issue from a particular viewpoint.

Defect Removal Efficiency

Related to defect containment, but this time measuring defects removed. defect removal efficiency = removed defects / total defects.

What makes a measurement valid?

Representativeness: - Whatever is being measured should be representative of the phenomenon about which one seeks to make an observation. DANGER: - Wrong measurement can lead to wrong conclusions.

Requirements Stability

Requirements Stability = initial requirements / total requirements.

Simplicity is deceiving

Some things are very difficult to measure. - Remember - what is the "quality" of a piece of software? - How "big" is a computer program? - Number of lines of code? - Number of functions? - What about comments? - Test cases?

Why Measure?

We can use numbers to reason about the things around us. We can take sample measurements and infer properties. - e.g Observe response-times of a system for an hour, and make inference about general response times.

An Empirical Relations System

We have sets of entities and relations. To measure these, we need to map the entities onto a scale.

Object-Oriented Metric

Weighted Method per Class (WMC): - Average of the Cyclomatic Complexity scores for all methods in a class. Depth of Inheritance Tree: - What is the maximum number of levels of class inheritance. Many others - selection depends (as always) on the goals, and question.

Scale - Ratio

measurement is the estimation of the ratio from current magnitude to a unit. e.g length. Measures can be compared by ratio. Choice of unit is arbitrary, but the choice of zero is fixed. - Zero represents a complete absence of all units.