Software review
Encyclopedia
A software review is "A process or meeting during which a software product is examined by a project personnel, managers, users, customers, user representatives, or other interested parties for comment or approval".
In this context, the term "software product" means "any technical document or partial document, produced as a deliverable of a software development activity", and may include documents such as contracts, project plans and budgets, requirements documents, specifications, designs, source code, user documentation, support and maintenance documentation, test plans, test specifications, standards, and any other type of specialist work product.
.
Research studies tend to support the conclusion that formal reviews greatly outperform informal reviews in cost-effectiveness. Informal reviews may often be unnecessarily expensive (because of time-wasting through lack of focus), and frequently provide a sense of security which is quite unjustified by the relatively small number of real defects found and repaired.
process originally developed at IBM by Michael Fagan
. Differing types of review may apply this structure with varying degrees of rigour, but all activities are mandatory for inspection:
A second, but ultimately more important, value of software reviews is that they can be used to train technical authors in the development of extremely low-defect documents, and also to identify and remove process inadequacies that encourage defects (the defect prevention process).
This is particularly the case for peer reviews
if they are conducted early and often, on samples of work, rather than waiting until the work has been completed. Early and frequent reviews of small work samples can identify systematic errors in the Author's work processes, which can be corrected before further faulty work is done. This improvement in Author skills can dramatically reduce the time it takes to develop a high-quality technical document, and dramatically decrease the error-rate in using the document in downstream processes.
As a general principle, the earlier a technical document is produced, the greater will be the impact of its defects on any downstream activities and their work products. Accordingly, greatest value will accrue from early reviews of documents such as marketing plans, contracts, project plans and schedules, and requirements specifications. Researchers and practitioners have shown the effectiveness of reviewing process in finding bugs and security issues,.
In this context, the term "software product" means "any technical document or partial document, produced as a deliverable of a software development activity", and may include documents such as contracts, project plans and budgets, requirements documents, specifications, designs, source code, user documentation, support and maintenance documentation, test plans, test specifications, standards, and any other type of specialist work product.
Varieties of software review
Software reviews may be divided into three categories:- Software peer reviewSoftware peer reviewIn software development, peer review is a type of software review in which a work product is examined by its author and one or more colleagues, in order to evaluate its technical content and quality.-Purpose:...
s are conducted by the author of the work product, or by one or more colleagues of the author, to evaluate the technical content and/or quality of the work.
- Software management reviews are conducted by management representatives to evaluate the status of work done and to make decisions regarding downstream activities.
- Software audit reviewSoftware audit reviewA software audit review, or software audit, is a type of software review in which one or more auditors who are not members of the software development organization conduct "An independent examination of a software product, software process, or set of software processes to assess compliance with...
s are conducted by personnel external to the software project, to evaluate complianceCompliance (regulation)In general, compliance means conforming to a rule, such as a specification, policy, standard or law. Regulatory compliance describes the goal that corporations or public agencies aspire to in their efforts to ensure that personnel are aware of and take steps to comply with relevant laws and...
with specifications, standards, contractual agreements, or other criteria.
Different types of reviews
- Code reviewCode reviewCode review is systematic examination of computer source code. It is intended to find and fix mistakes overlooked in the initial development phase, improving both the overall quality of software and the developers' skills...
is systematic examination (often as peer reviewSoftware peer reviewIn software development, peer review is a type of software review in which a work product is examined by its author and one or more colleagues, in order to evaluate its technical content and quality.-Purpose:...
) of computer source code. - Pair programmingPair programmingPair programming is an agile software development technique in which two programmers work together at one workstation. One, the driver, types in code while the other, the observer , reviews each line of code as it is typed in...
is a type of code review where two persons develop code together at the same workstation. - InspectionSoftware inspectionInspection in software engineering, refers to peer review of any work product by trained individuals who look for defects using a well defined process...
is a very formal type of peer review where the reviewers are following a well-defined process to find defects. - WalkthroughSoftware walkthroughIn software engineering, a walkthrough or walk-through is a form of software peer review "in which a designer or programmer leads members of the development team and other interested parties through a software product, and the participants ask questions and make comments about possible errors,...
is a form of peer review where the author leads members of the development team and other interested parties through a software product and the participants ask questions and make comments about defects. - Technical reviewSoftware technical reviewA software technical review is a form of peer review in which "a team of qualified personnel ... examines the suitability of the software product for its intended use and identifies discrepancies from specifications and standards. Technical reviews may also provide recommendations of alternatives...
is a form of peer review in which a team of qualified personnel examines the suitability of the software product for its intended use and identifies discrepancies from specifications and standards.
Formal versus informal reviews
"Formality" identifies the degree to which an activity is governed by agreed (written) rules. Software review processes exist across a spectrum of formality, with relatively unstructured activities such as "buddy checking" towards one end of the spectrum, and more formal approaches such as walkthroughs, technical reviews, and software inspections, at the other. IEEE Std. 1028-1997 defines formal structures, roles, and processes for each of the last three ("formal peer reviews"), together with software auditsSoftware quality assurance
Software quality assurance consists of a means of monitoring the software engineering processes and methods used to ensure quality. The methods by which this is accomplished are many and varied, and may include ensuring conformance to one or more standards, such as ISO 9000 or a model such as...
.
Research studies tend to support the conclusion that formal reviews greatly outperform informal reviews in cost-effectiveness. Informal reviews may often be unnecessarily expensive (because of time-wasting through lack of focus), and frequently provide a sense of security which is quite unjustified by the relatively small number of real defects found and repaired.
IEEE 1028 generic process for formal reviews
IEEE Std 1028 defines a common set of activities for "formal" reviews (with some variations, especially for software audit). The sequence of activities is largely based on the software inspectionSoftware inspection
Inspection in software engineering, refers to peer review of any work product by trained individuals who look for defects using a well defined process...
process originally developed at IBM by Michael Fagan
Fagan inspection
Fagan inspection refers to a structured process of trying to find defects in development documents such as programming code, specifications, designs and others during various phases of the software development process...
. Differing types of review may apply this structure with varying degrees of rigour, but all activities are mandatory for inspection:
- 0. [Entry evaluation]: The Review Leader uses a standard checklist of entry criteria to ensure that optimum conditions exist for a successful review.
- 1. Management preparation: Responsible management ensure that the review will be appropriately resourced with staff, time, materials, and tools, and will be conducted according to policies, standards, or other relevant criteria.
- 2. Planning the review: The Review Leader identifies or confirms the objectives of the review, organises a team of Reviewers, and ensures that the team is equipped with all necessary resources for conducting the review.
- 3. Overview of review procedures: The Review Leader, or some other qualified person, ensures (at a meeting if necessary) that all Reviewers understand the review goals, the review procedures, the materials available to them, and the procedures for conducting the review.
- 4. [Individual] Preparation: The Reviewers individually prepare for group examination of the work under review, by examining it carefully for anomalies (potential defects), the nature of which will vary with the type of review and its goals.
- 5. [Group] Examination: The Reviewers meet at a planned time to pool the results of their preparation activity and arrive at a consensus regarding the status of the document (or activity) being reviewed.
- 6. Rework/follow-up: The Author of the work product (or other assigned person) undertakes whatever actions are necessary to repair defects or otherwise satisfy the requirements agreed to at the Examination meeting. The Review Leader verifies that all action items are closed.
- 7. [Exit evaluation]: The Review Leader verifies that all activities necessary for successful review have been accomplished, and that all outputs appropriate to the type of review have been finalised.
Value of reviews
The most obvious value of software reviews (especially formal reviews) is that they can identify issues earlier and more cheaply than they would be identified by testing or by field use (the defect detection process). The cost to find and fix a defect by a well-conducted review may be one or two orders of magnitude less than when the same defect is found by test execution or in the field.A second, but ultimately more important, value of software reviews is that they can be used to train technical authors in the development of extremely low-defect documents, and also to identify and remove process inadequacies that encourage defects (the defect prevention process).
This is particularly the case for peer reviews
Software peer review
In software development, peer review is a type of software review in which a work product is examined by its author and one or more colleagues, in order to evaluate its technical content and quality.-Purpose:...
if they are conducted early and often, on samples of work, rather than waiting until the work has been completed. Early and frequent reviews of small work samples can identify systematic errors in the Author's work processes, which can be corrected before further faulty work is done. This improvement in Author skills can dramatically reduce the time it takes to develop a high-quality technical document, and dramatically decrease the error-rate in using the document in downstream processes.
As a general principle, the earlier a technical document is produced, the greater will be the impact of its defects on any downstream activities and their work products. Accordingly, greatest value will accrue from early reviews of documents such as marketing plans, contracts, project plans and schedules, and requirements specifications. Researchers and practitioners have shown the effectiveness of reviewing process in finding bugs and security issues,.