Many people think that the main benefit from Inspection is in identifying and removing Major defects early (e.g. before source code reaches test phases). This is not the case. (My experience is that Inspection is as bad as testing in % defect-removal effectiveness. In very rough terms half of every defect present is not identified or removed.) The really important economic effect of Inspection is not what happens at the level of a single document, but in teaching the people and the organization. The real effect of Inspection is in:

• teaching individual engineers exactly how often they violate best practice rules
   

• motivating the engineers to take rules seriously (really avoid injecting Major defects)
   

• regulating flow of documentation, so that high Major defect documents can neither exit nor enter adjacent work processes.

 Staff involved in Inspections learn very quickly how to stop injecting defects. Typically, the defects introduced by an author reduce at the rate of about 50% every time a new document is written and Inspected. For example, using Inspection, Raytheon reduced ‘rework’ costs, as a % of development costs, from 43% to 5% in an eight year period [DION95].


Sampling
One other little-appreciated aspect of Inspection is that you can use it by sampling a small section of a large document, rather than trying to ‘clean up’ the entire document. If the sample shows a high Major defect density (say more than one Major/Page) then the document is probably ‘polluted’ and action can be taken to analyze the defect sources. A complete rewrite may be necessary using appropriate specification rules or new/improved source documents. This is generally cheaper than trying to clean up the entire document using defect removal Inspection or testing.

Principle 8.  Reuse what you learn about risk

Standards, rules and guidance must capture and assist good practice. Continuous process improvement is also needed.

In the previous section, the importance of Inspection was discussed and rules were highlighted as one of the essentials required to support it. It is worth emphasizing the aspect of reuse that is occurring here. The more effort that is put into making rules more effective and efficient by incorporating feedback from Inspections, the more productive the Inspections and the greater the reduction in risk.

Even more benefit can be achieved if what is learnt from Inspection is used to modify the processes that are causing the defects; Continuous Process Improvement has been shown to have a major influence on risk. For example, Raytheon has achieved zero deviation from plans and budgets over several years. They used a $1million/year (for 1,000 software engineers) for 8 years to do continuous software process improvement. They report that the return on this investment was $7.70 per $1 invested on improving processes such as requirements, testing and Inspection itself. Their software defect rate went down by a factor of three [DION95].

Using Inspection, analysis of the identified defects to find process improvements is carried out in the Defect Prevention Process (DPP). DPP was developed from 1983 at IBM by Robert Mays and Carole Jones and, is today recognized as the basis for SEI CMM Level Five. The breakthrough in getting DPP to work, compared to earlier failed efforts within IBM, was probably in the decentralization of analysis activity to many smaller groups, rather than one ‘Lab Wide’ effort by a Quality Manager. This follows what the quality Guru Dr. W Edwards Deming taught the Japanese; factory workers must analyze their own statistics and be empowered to improve their own work processes.

Analysis of ‘root causes’ of defects is very much a risk analysis effort [HALL98] and a handful of my clients are reporting success at doing so. But, most are still working on other disciplines like Inspection and others elsewhere in this paper.


Principle 9. Delegate personal responsibility for risk

People must be give personal responsibility in their sector for identification and mitigation of risks.

To back up communicating about risk, people must be given ownership of the risks in their sector (e.g. allocating ownership/sign off of IE tables and giving people specific roles within Inspections).


Principle 10. Contract out risk

Make vendors contractually responsible for risks, they will give you better advice and services as a result.

I would like to point out that contracting for products and services gives great opportunity to legally and financially control risks by squarely putting them on someone else’s shoulders.

The effect of contracting a risk to someone else is that:

• you have gotten rid of the risk in some senses, but if they fail, you will still be affected!
   
• the supplier (assuming they get the risk) will be more motivated to take steps to eliminate the risks
   
• be motivated to tell you exactly what you have to do the avoid being hit by risks
   
• might come up with a more realistic bid and time plan to cope with the risks.

Summary

Risks can be handled in many ways and at many levels. I have tried to point out some risk management methods which are not so well known or well treated in existing literature. Hopefully, the need to fully integrate risk management into all the development and maintenance processes is clear.

Table 6 and Table 7 recap the ideas presented in this paper. Table 6 is a set of policies for risk management [See GILB98 for more detail]. Table 7 contains ‘Twelve Tough Questions’ to ask when assessing risk.

more...