Software Engineering chapter 24
Importance of standards
Encapsulation of best practice- avoids repetition of past mistakes. They are a framework for defining what quality means in a particular setting i.e. that organization's view of quality. They provide continuity - new staff can understand the organisation by understanding the standards that are used.
Formal Review Meeting
Between three and five people (typically) should be involved in the review. Advance preparation should occur but should require no more than two hours of work for each person. The duration of the review meeting should be less than two hours. Focus is on a work product (e.g., a portion of a requirements model, a detailed component design, source code for a component)
Definitions: Measures, Metrics and Indicators
A measure provides a quantitative indication of the extent, amount, dimension, capacity, or size of some attribute of a product or process The IEEE glossary defines a metric as "a quantitative measure of the degree to which a system, component, or process possesses a given attribute." An indicator is a metric or combination of metrics that provide insight into the software process, a software project, or the product itself
Quality planning
A quality plan sets out the desired product qualities and how these are assessed and defines the most significant quality attributes. The quality plan should define the quality assessment process. It should set out which organisational standards should be applied and, where necessary, define new standards to be used.
ISO 9001 standards framework
An international set of standards that can be used as a basis for developing quality management systems. ISO 9001, the most general of these standards, applies to organizations that design, develop and maintain products, including software. The ISO 9001 standard is a framework for developing software standards. It sets out general quality principles, describes quality processes in general and lays out the organizational standards and procedures that should be defined. These should be documented in an organizational quality manual.
Shared good practice
Check before check-in Programmers are responsible for organizing their own code reviews with other team members before the code is checked in to the build system. Never break the build Team members should not check in code that causes the system to fail. Developers have to test their code changes against the whole system and be confident that these work as expected. Fix problems when you see them If a programmer discovers problems or obscurities in code developed by someone else, they can fix these directly rather than referring them back to the original developer.
Component-Level Design Metrics
Cohesion metrics: a function of data objects and the locus of their definition Coupling metrics: a function of input and output parameters, global variables, and modules called Complexity metrics: hundreds have been proposed (e.g., cyclomatic complexity)
What to look for in reviews
Errors and defects (common definitions) Error —a quality problem found before the software is released to end users Defect —a quality problem found only after the software has been released to end-users
Software quality management
Concerned with ensuring that the required level of quality is achieved in a software product. Three principal concerns: At the organizational level, quality management is concerned with establishing a framework of organizational processes and standards that will lead to high-quality software. At the project level, quality management involves the application of specific quality processes and checking that these planned processes have been followed. At the project level, quality management is also concerned with establishing a quality plan for a project. The quality plan should set out the quality goals for the project and define what processes and standards are to be used.
McCabe's Cyclomatic Complexity
Cyclomatic complexity is a software metric (measurement). It was developed by Thomas J. McCabe, Sr. in 1976 and is used to indicate the complexity of a program. It is a quantitative measure of logical strength of the program. It directly measures the number of linearly independent paths through a program's source code. Cyclomatic complexity is computed using the control flow graph of the program: the nodes of the graph correspond to indivisible groups of commands of a program, and a directed edge connects two nodes if the second command might be executed immediately after the first command. Mathematically, the cyclomatic complexity of a structured program is defined with reference to the control flow graph of the program, a directed graph containing the basic blocks of the program, with an edge between two basic blocks if control may pass from the first to the second. The complexity M is then defined as M = E − N + 2P Where E = the number of edges of the graph. N = the number of nodes of the graph. P = the number of connected components (exit nodes).
Measurement Process
Formulation. The derivation of software measures and metrics appropriate for the representation of the software that is being considered. Collection. The mechanism used to accumulate data required to derive the formulated metrics. Analysis. The computation of metrics and the application of mathematical tools. Interpretation. The evaluation of metrics results in an effort to gain insight into the quality of the representation. Feedback. Recommendations derived from the interpretation of product metrics transmitted to the software team.
Software fitness for purpose
Has the software been properly tested? Is the software sufficiently dependable to be put into use? Is the performance of the software acceptable for normal use? Is the software usable? Is the software well-structured and understandable? Have programming and documentation standards been followed in the development process?
Types of Reviews
Informal reviews Simple desk check of a software engineering work product with a colleague Casual meeting (involving more than 2 people) for the purpose of reviewing a work product Side effect of pair programming Formal reviews Walkthroughs Inspections
Standards development
Involve practitioners in development. Engineers should understand the rationale underlying a standard. Review standards and their usage regularly. Standards can quickly become outdated and this reduces their credibility amongst practitioners. Detailed standards should have specialized tool support. Excessive clerical work is the most significant complaint against standards. Web-based forms are not good enough.
Software standards
Standards define the required attributes of a product or process. They play an important role in quality management. Standards may be international, national, organizational or project standards.
Quality conflicts
It is not possible for any system to be optimized for all of these attributes - for example, improving robustness may lead to loss of performance. The quality plan should therefore define the most important quality attributes for the software that is being developed. The plan should also include a definition of the quality assessment process, an agreed way of assessing whether some quality, such as maintainability or robustness, is present in the product.
Pair programming weaknesses
Mutual misunderstandings Both members of a pair may make the same mistake in understanding the system requirements. Discussions may reinforce these errors. Pair reputation Pairs may be reluctant to look for errors because they do not want to slow down the progress of the project. Working relationships The pair's ability to discover defects is likely to be compromised by their close working relationship that often leads to reluctance to criticize work partners.
Object-Oriented Metrics
Number of scenario scripts (use-cases) Number of support classes (required to implement the system but are not immediately related to the problem domain) Average number of support classes per key class (analysis class) Number of subsystems (an aggregation of classes that support a function that is visible to the end-user of a system)
Knots
Number of times control paths cross private int search (PhoneEntry key) { for (int loc = 0; loc < entries.length; loc ++) { if (entries[loc].equals(key)) return loc; } return NOT_FOUND; One knot above
Who attends formal reviews
Producer —the individual who has developed the work product informs the project leader that the work product is complete and that a review is required Review leader —evaluates the product for readiness, generates copies of product materials, and distributes them to two or three reviewers for advance preparation. Reviewer(s) —expected to spend between one and two hours reviewing the product, making notes, and otherwise becoming familiar with the work. Recorder —reviewer who records (in writing) all important issues raised during the review.
Why Opt for FP?
Programming language independent Used readily countable characteristics that are determined early in the software process Does not "penalize" inventive (short) implementations that use fewer LOC that other more clumsy versions Makes it easier to measure the impact of reusable components
Quality management and agile development
Quality management in agile development is informal rather than document-based. It relies on establishing a quality culture, where all team members feel responsible for software quality and take actions to ensure that quality is maintained. The agile community is fundamentally opposed to what it sees as the bureaucratic overheads of standards-based approaches and quality processes as embodied in ISO 9001.
Quality management activities
Quality management provides an independent check on the software development process. The quality management process checks the project deliverables to ensure that they are consistent with organizational standards and goals The quality team should be independent from the development team so that they can take an objective view of the software. This allows them to report on software quality without being influenced by software development issues.
Quality culture
Quality managers should aim to develop a 'quality culture' where everyone responsible for software development is committed to achieving a high level of product quality. They should encourage teams to take responsibility for the quality of their work and to develop new approaches to quality improvement. They should support people who are interested in the intangible aspects of quality and encourage professional behavior in all team members.
Quality plans
Quality plan structure Product introduction; Product plans; Process descriptions; Quality goals; Risks and risk management. Quality plans should be short, succinct documents If they are too long, no one will read them.
ISO 9001 certification
Quality standards and procedures should be documented in an organisational quality manual. An external body may certify that an organisation's quality manual conforms to ISO 9000 standards. Some customers require suppliers to be ISO 9000 certified although the need for flexibility here is increasingly recognised.
Software quality
Quality, simplistically, means that a product should meet its specification. There is a tension between customer quality requirements (efficiency, reliability, etc.) and developer quality requirements (maintainability, reusability, etc.); Some quality requirements are difficult to specify in an unambiguous way; Software specifications are usually incomplete and often inconsistent.
Conducting the Review
Review the product, not the producer. Set an agenda and maintain it. Limit debate. Enunciate problem areas, but do not solve problems noted. Take written notes. Limit the number of participants and insist upon advance preparation. Develop a checklist for each product that is likely to be reviewed. Allocate resources and schedule time for FTRs. Conduct meaningful training for all reviewers.
Function-Based Metrics
The function point metric (FP) first proposed by Albrecht [ALB79], can be used effectively as a means for measuring the functionality delivered by a system. Function points are derived using an empirical relationship based on countable (direct) measures of software's information domain and assessments of software complexity Information domain values are defined in the following manner: number of external inputs (EIs) number of external outputs (EOs) number of external inquiries (EQs) number of internal logical files (ILFs) number of external interface files (EIFs)
Pair programming
This is an approach where 2 people are responsible for code development and work together to achieve this. Code developed by an individual is therefore constantly being examined and reviewed by another team member. Pair programming leads to a deep knowledge of a program, as both programmers have to understand the program in detail to continue development. This depth of knowledge is difficult to achieve in inspection processes and pair programming can find bugs that would not be discovered in formal inspections
Process and product quality
The quality of a developed product is influenced by the quality of the production process. This is important in software development as some product quality attributes are hard to assess. However, there is a very complex and poorly understood relationship between software processes and product quality. The application of individual skills and experience is particularly important in software development; External factors such as the novelty of an application or the need for an accelerated development schedule may impair product quality.
Reviews and agile methods
The review process in agile software development is usually informal. In Scrum,, there is a review meeting after each iteration of the software has been completed (a sprint review), where quality issues and problems may be discussed. In Extreme Programming, pair programming ensures that code is constantly being examined and reviewed by another team member.
Non-functional characteristics
The subjective quality of a software system is largely based on its non-functional characteristics. This reflects practical user experience - if the software's functionality is not what is expected, then users will often just work around this and find other ways to do what they want to do. However, if the software is unreliable or too slow, then it is practically impossible for them to achieve their goals.
Basics on Measurement Theory
There are 2 forms of measurements Direct Used to directly measure some property or attribute Examples Thermometer - measure temperature Pedometer - measure the number of steps taken Indirect Measure one or more attributes to measure or infer some attribute Example Pedometer and Watch - number of steps taken in a given time interval is used to infer speed
Reviews
They are a meeting conducted by technical people for technical people a technical assessment of a work product created during the software engineering process a software quality assurance mechanism a training ground
Problems with standards
They may not be seen as relevant and up-to-date by software engineers. They often involve too much bureaucratic form filling. If they are unsupported by software tools, tedious form filling work is often involved to maintain the documentation associated with the standards.
Metrics - Errors
Total errors found Errtot = Errminor + Errmajor Where Minor errors found, Errminor —the number of errors found that can be categorized as minor (requiring less than some pre-specified effort to correct) Major errors found, Errmajor — the number of errors found that can be categorized as major (requiring more than some pre-specified effort to correct)
Metrics - Effort
Total review effort Ereview = Ep + Ea + Er Where Preparation effort, Ep —the effort (in person-hours) required to review a work product prior to the actual review meeting Assessment effort, Ea — the effort (in person-hours) that is expending during the actual review Rework effort, Er — the effort (in person-hours) that is dedicated to the correction of those errors uncovered during the review
V & V
Verification refers to the set of tasks that ensure that software correctly implements a specific function. Validation refers to a different set of tasks that ensure that the software that has been built is traceable to customer requirements. Boehm [Boe81] states this another way: Verification: "Are we building the product right?" Validation: "Are we building the right product?"
Software Measurements
We aim to measure certain attributes Size Complexity Understandability Maintainability Testability
Error Density
What is the percentage of errors in the work product? Error density represents the errors found per unit of work product reviewed. Error density = Errtot / WPS Where Work product size, WPS —a measure of the size of the work product that has been reviewed (e.g., the number of UML models, or the number of document pages, or the number of lines of code)
Agile QM and large systems
When a large system is being developed for an external customer, agile approaches to quality management with minimal documentation may be impractical. If the customer is a large company, it may have its own quality management processes and may expect the software development company to report on progress in a way that is compatible with them. Where there are several geographically distributed teams involved in development, perhaps from different companies, then informal communications may be impractical. For long-lifetime systems, the team involved in development will change Without documentation, new team members may find it impossible to understand development.
Why Do We Measure
assess the status of an ongoing project track potential risks uncover problem areas before they go "critical," adjust work flow or tasks, evaluate the project team's ability to control quality of software work products.
Project Metrics
used to minimize the development schedule by making the adjustments necessary to avoid delays and mitigate potential problems and risks used to assess product quality on an ongoing basis and, when necessary, modify the technical approach to improve quality. every project should measure: inputs—measures of the resources (e.g., people, tools) required to do the work. outputs—measures of the deliverables or work products created during the software engineering process. results—measures that indicate the effectiveness of the deliverables.