Some Software Process Improvement References

compiled and abstracted by Karl E. Wiegers

Diaz, Michael, and Jeff King. "How CMM Impacts Quality, Productivity, Rework, and the Bottom Line," CrossTalk, vol. 15, no. 3 (March 2002), pp. 9-14. Data on results of CMM maturity level on General Dynamics Decision System's improvement trends for rework, defect phase containment, predicted quality, and productivity. From level 2 to level 5, rework percent dropped from 23.2% to 6.8%; phase containment effectiveness rose from 25.5% to 87.3%; customer reported unique defects (CRUD) density dropped from 3.20 to 0.19/KSLOC, and productivity rose from 1X to 2.9X.

Mogilensky, Judah. "The Human Side of Process Improvement," STQE, vol. 3, no. 6 (November/December 2001), pp. 43-47. Describes how SPI also requires changes in human dynamics from closed to open and changes in personal interactions.

Kimmerly, Paul, Joel Ogren, Carol Mullins, Shannon Morten, Pamela Yancey, and Wendell Bazemore. "Perspectives on the Software Engineering Process Group," CrossTalk, vol. 14, no. 3 (March 2001), pp. 4-8. Views of the role and activities of the SEPG from senior managers, experienced SEPG members, and new SEPG members.

Frailey, Dennis. "Reducing Cycle Time," Software Development, vol. 8, no. 8 (August 2000), pp. 78-80. Nice description of causes for cycle time delays, identifying bottlenecks in the process, identifying the value stream. Cycle time principles: cycles of learning principle, small batch principle, smooth flow principle. Make sure you don't just shorten the longest step in the development process or eliminate steps (locally optimize) that add strategic value (globally optimize instead).

Jacobsen, J. James. "Case Study: Selling the CMM", Software Testing and Quality Engineering, vol. 2, no. 2 (March/April 2000), pp. 32-39. Description of how CMM was sold and implemented at a financial services company. Good tips and story about how they organized for SPI.

O'Connell, Emilie, and Hossein Saiedian. "Can You Trust Software Capability Evaluations?", IEEE Computer, (Feb. 2000), pp. 28-35. Overview of SCE team activities, issues, and shortcomings which can lead to misleading results.

Wiegers, Karl E., and Doris Sturzenberger. "A Modular Software Process Mini-Assessment Method", IEEE Software, vol. 17, no. 1 (January/February 2000), pp. 62-69. Describes a mini-assessment "toolkit" that can be assembled in various ways to meet the process improvement needs of different projects. Some data reported on experiences with 24 mini-assessments.

Maxwell, Katriina D., and Pekka Forselius, "Benchmarking Software Development Productivity," IEEE Software, vol. 17, no. 1 (January/February 2000), pp. 80-88. Evaluated large number of variables to determine factors having the largest impact on productivity from 206 projects at 26 companies in Finland. Requirements volatility was a major contributor for insurance, banking, manufacturing industries.

Laporte, Claude Y., and Sylvie Trudel. "Addressing People Issues when Developing and Implementing Engineering Processes", CrossTalk, vol. 12, no. 11 (November 1999), pp. 8-13. In addition to 16 useful lessons learned, Table 1 has a nice list of meeting guidelines for any kind of meeting.

Kelley, Declan P., and Bill Culleton. "Process Improvement for Small Organizations", IEEE Computer, vol. 32, no. 10 (October 1999), pp. 41-47. Case study of applying CMM at a fairly small organization.

Rothman, Johanna. "A Problem-Based Approach to Software Process Improvement: A Case Study," CrossTalk, vol. 12, no. 10 (October 1999), pp. 23-27. Case study of developing a process improvement process.

Natwick, Gary, Geoff Draper, and Lennis Bearden. "Software Mini-Assessments: Process and Practice." CrossTalk, vol. 12, no. 10 (October 1999), pp. 10-13. Application of Motorola mini-assessment method to Harris Information Systems Division.

Lochner, Jane T. "16 Critical Software Practices for Performance-Based Management," CrossTalk, vol. 12, no. 10 (October 1999), pp. 6-9. Lists 16 best practices from Airlie Council, in categories of Project Control, Product Construction, and Product Integrity.

Hadden, Rita. "Building Highly Effective SPI Projects: What You Must Do Right," Cutter IT Journal, vol. 12, no. 9 (September 1999), pp. 10-16. Key SPI activities include: openness to learning from what didn't work in other organizations doing SPI; insistence on a commitment-oriented software culture; focusing on "quick win" practices; providing ongoing structured mentoring to IT staff; role modeling by IT executives, managers, and project leaders who hold each other accountable; software engineering process groups skilled at teamwork and culture change management strategies; consistent policy enforcement by management; simple but effective project metrics; a creative incentive and reward system; release management for process refinement

Brown, Norm. "High-Leverage Best Practices: What Hot Companies Are Doing to Stay Ahead", Cutter IT Journal, vol. 12, no. 9 (September 1999), pp. 4-9. Update of Airlie Council best practices, now with 16 grouped into 3 categories: product integrity (inspect requirements and design; manage testing as a continuous process; compile and smoke test frequently), product construction (adopt life cycle configuration management; manage and trace requirements; use system-based software design; ensure data and database interoperability; define and control interfaces; design twice, code once; assess reuse risks and costs), and project control (adopt a program risk management process; estimate empirically cost and schedule; use metrics to manage; track earned value; track defects against quality targets; treat people as the most important resource).

Johnson, Mark. Software Testing and Quality Engineering, vol. 1, no. 4 (July/August 1999), pp.44-50. Perspectives on how to make the quest for ISO 9000 registration fit your organization's objectives.

Paulk, Mark C. "Using the Software CMM with Good Judgment," Software Quality Professional, vol. 1, no. 3 (June 1999), pp. 19-29. Explanation of how to interpret KPAs for small organizations and small projects and where the CMM applies.

Ben-Yaacov, Giora, and Arun Joshi. "Investing in Quality Does Indeed Pay: A Software Process Improvement Case Study," Software Quality Professional, vol. 1, no. 3 (June 1999), pp. 45-53. Describes SPI approach for an Indian company leading to ISO 9001 registration. Detailed reporting of costs of SPI and ROI, amounting to 4.5 ROI for 1 year, estimated 15.9 for 5 years.

Dutta, Soumitra, Michael Lee, and Luk Van Wassenhove. “Software Engineering in Europe: A study of Best Practices”, IEEE Software, vol. 16, no. 3 (May/June 1999), pp. 82-90. Study of the extent to which some 42 best practices are being applied in almost 400 companies in 20 European countries, by process category and business sector. Very wide ranges, but averages are around 51% adoption of industry best software practices.

Marshall, Richard M. "Manage Development Better with CMM", Enterprise Development, (May 1999), pp. 42-50. Interesting summary of CMM, written from a fairly practical perspective.

Grable, Ross, Jacquelyn Jernigan, Casey Pogue, and Dale Divis. "Metrics for Small Projects: Experiences at the SED," IEEE Software, vol. 16., no. 2 (March/April 1999), pp. 21-29. Use of metrics and CMM on 10–50 KLOC embedded systems. Measured 3X productivity improvement going from level 1 to level 2.

Houston, Dan. "Cost of Software Quality: Justifying Software Process Improvement to Managers," Software Quality Professional, vol. 1, no. 2 (March 1999), pp. 8-16. Just what it says. Good references.

Wiegers, Karl E. "Why is Process Improvement So Hard?", Software Development, Web edition (www.sdmagazine.com), (February, 1999). Five reasons software process improvement is so difficult to pull off: not enough time, lack of knowledge, wrong motivations, dogmatic approaches, and insufficient commitment.

Webb, David, and Watts S. Humphrey. "Using the TSP on the TaskView Project," CrossTalk, vol. 12, no. 2 (February 1999), pp. 3-10. Case study of the team software process used on a smallish (<30 KLOC) project. Big benefits in reducing testing time, overall small increase in productivity.

Curtis, Bill. "Which Comes First: the Organization or Its Processes?", IEEE Software, vol. 15, no. 6 (November/December 1998), pp. 10-13. Nice story about importance of culture, problems of checklist mentality for SPI, staged vs. continuous process standards.

Gray, Lewis. "Why Coaches are Needed in Software Process Improvement," CrossTalk, vol. 11., no. 9 (September 1998), pp. 16-19. Describes SPI roles of coach, sponsor, champion, and change agent. Qualities of a good coach: knowledge of the game, sense of humor, honesty, inspires trust, good communication, respect for team, no grudges, negative reactions seen in the proper perspective, focus on what can be done.

Bamford, Robert C., and William J. Deibler, II. "Hybrid Multi-Model Assessment: When the CMM Meets ISO 9001," CrossTalk, vol. 11., no. 9 (September 1998), pp. 3-8. Strategy and methods to employ formal appraisals to determine which model, or which elements from either model, offer the most value for a particular org.

Humphrey, Watts S. "Three Dimensions of Process Improvement. Part III: The Team Process", CrossTalk, vol. 11, no. 4 (April 1998), pp. 14-17. Overview of the Team Software Process, mapping to KPAs, early results.

Orville, Douglas D. "Process Action Teams: From ‘Black Holes’ to ‘Shining Stars’," CrossTalk, vol. 11, no. 2 (February 1998), pp. 18-21. Describes model for how to operate PATS. Suggests PAT members work only during working sessions of 1.5-3.0 days every ~2 weeks.

Hadden, Rita. "How Scalable Are CMM Key Practices?", CrossTalk, vol. 11, no. 4 (April 1998), pp. 18-23. Case study of convincing people to use Level 2 and 3 practices on a specific project.

Laitinen, Mauri, and Mohamed E. Fayad. "Surviving a Process Performance Crash." Communications of the ACM, vol. 41, no. 2 (Feb 1998), pp. 83-85. Process metrics; measure processes rather than people; a process evaluation example; endless change means more control; more processes means greater creativity.

Klein, David C. "If You Get Straight A’s, You Must be Intelligent: Respecting the Intent of the Capability Maturity Model," CrossTalk, vol. 11, no. 2 (February 1998), pp. 22-23. The spirit of the CMM and indicators that the SCE rating is valid (proper staffing of process improvement teams, processes-based project or organizational value, quantify ROI of process improvement, use external experts for the right reasons, define process improvement in SOW).

Bach, James. "Microdynamics of Process Evolution." IEEE Computer (Feb. 1998), pp. 111-113. Small things become important in process change.

Jones, Capers. "Polishing the Software Process," Software Development, vol. 5, no. 12 (December 1997), pp. 47-54. Use of cost accounting method to measure costs and likely returns from SPI. Describes Jones's 6-step sequence for SPI.

Diaz, Michael, and Joseph Sligo. "How Software Process Improvement Helped Motorola," IEEE Software, vol. 14, no. 5 (September/October 1997),pp. 75-81. Describes Motorola approach to SPI in one division, and quantifies impact on quality, cycle time, and productivity as a function of maturity level of individual projects. Calculated ROI from SPI of 6.8, and describes how ROI was calculated.

Puffer, Joe. "Action Planning." Software Process Newsletter, No. 9 (Spring 1997), pp. 7-10. Approach to doing large-scale action planning following a full assessment.

Dymond, Ken. "Essence and Accidents in SEI-Style Assessments, or 'Maybe This Time the Voice of the Engineer Will Be Heard'." Software Process Newsletter, No. 9 (Spring 1997), pp. 1-7. Issues around having successful assessments, in context of SPA/CBA IPI.

Statz, Joyce, Don Oxley, and Patrick O'Toole. "Identifying and Managing Risks for Software Process Improvement," CrossTalk (April 1997),pp. 13-18. Describes common risk factors for SPI and results of analyzing high and low severity risks on several SPI programs (successful and other).

Haley, Thomas J. "Software Process Improvement at Raytheon," IEEE Software, vol. 13, no. 6 (November 1996), pp. 33-41. Update of the Raytheon experience, describing their SEPG and SPI organization, major leverage points that gave large benefits (participation in system definition, requirements definition, inspections, integration and qualification testing), and 8+ years of Cost of Quality data showing reduction of rework, productivity increase, reduced defect density, and improved cost predictability.

Konrad, Michael D., and Mark. C. Paulk. "An Overview of SPICE's Model for Process Management." Proceedings of 5th International Conference on Software Quality.

Lloyd, Peter. "Return on Investment from Continuous Software Process Improvements in an International Company. Proceedings of 5th International Conference on Software Quality. Schlumberger experience.

Johnson, Donna L., and Judith G. Brodman. "Realities and Rewards of Software Process Improvement," IEEE Software, vol. 13, no. 6 (November 1996), pp.99-101. Results of a study on which KPAs positively impact product, quality, and schedule, and therefore which activities have the greatest ROI.

Brown, Norm. "Industrial-Strength Management Strategies," IEEE Software, vol. 13, no. 4 (July 1996), pp. 94-103. States 9 principal best practices for managing large projects (from Airlie Council): risk mgmt, agreement on interfaces, formal inspections, metrics based schedule and management, binary quality gates at inch pebble level, program wide visibility of progress vs. plan,efect tracking against quality targets, configuration mgmt, and people aware management accountability.

Wiegers, Karl. "Misconceptions of the CMM," Software Development, vol. 4, no. 11 (November 1996), pp. 57-64. 14 common misconceptions about the software CMM are clarified.

Wiegers, Karl. "Software Process Improvement:10 Traps to Avoid," Software Development, vol. 4, no. 5 (May 1996), pp. 51-58. Ten symptoms of process improvement activities going wrong, and some possible solutions.

Masters, Steve. "CMM Appraisal Framework, Version 1.0," Software Process Improvement Forum, vol. 3, no. 1 (Jan/Feb 1996), pp. 18-20. Describes CAF: appraisal architecture, CAF requirements, CAF compliance.

Brodman, Judith G., and Donna L. Johnson. "Return on Investment (ROI) from Software Process Improvement as Measured by US Industry," Software Process Improvement and Practice, vol. 1, no. 1, pp. 35-47. Describes ways in which industry measures ROI, with just a little data on specific ROI results obtained.

Livingston, Janis, Kellye Prosise, and Robert Altizer. "Process Improvement Matrix: A Tool for Measuring Progress Toward Better Quality." Proceedings of 5th International Conference on Software Quality. Use of the matrix at Motorola for monitoring process improvements and get results on quality on the current project.

Butler, Kelley L. "The Economic Benefits of Software Process Improvement," CrossTalk, 7/95, pp. 14-17. 4 projects at Oklahoma City Air Logistics Center showed 10X defect reduction (2nd project vs. first), 10X increase in productivity, 26% reduction in maintenance (last 2 years), ROI of 7.5 to 1 from SPI investment of $1.5M over 8 years (ROI based on assuming no productivity increase after project 1 if SPI had not been happening).

Wakulczyk, Marek. "NSSF Spot Check: A Metric Toward CMM Level 2," CrossTalk, 7/95, pp. 23-24. Use of Kiviat diagrams to track progress on CMM KPAs, based on responses to mini-assessment (spot check) between full assessments.

Quinn, David P. "A Revised Capability Maturity Model for Heretics," Software Process Improvement Forum, vol. 2, no. 3, (May/June 1995), pp. 10-14. Proposed new CMM with level between current 1 & 2, with new SW Test Management KPA, RM, SQA, and SCM; new Level 3 has PP, PT, SM, PE, and PR.

"TRILLIUM: A Model for the Assessment of Telecom Product Development & Support Capability," Software Process Newsletter, No. 2 (Winter 1995), pp. 3-8. Describes TRILLIUM process model. Derived from CMM, but has 8 capability areas that are different from CMM. Good description.

Paulk, Mark C. "How ISO 9001 Compares with the CMM," IEEE Software, vol. 12, no. 1 (January 1995), pp. 74-83. Detailed comparison of CMM and ISO 9001, showing how ISO clauses are related to components in CMM, and to what extent each KPA is satisfied by an ISO 9001-compliant organization.

Wohlwend, Harvey, and Susan Rosenbaum. "Schlumberger's Software Improvement Program," IEEE Transactions on Software Engineering, vol. 20, no. 11 (11/94), pp. 833-839. Describes corporate wide PI effort at Schlumberger over several years and benefits achieved.

Khajenoori, Soheil. "Process-Oriented Software Education," Quality Time column in IEEE Software, vol. 11, no. 6, 11/94, pp. 99-101.

Paulish, Daniel J., and Anita D. Carleton. "Case Studies of Software-Process-Improvement Measurement," IEEE Computer, vol.27, no.9 (9/94), pp. 50-57. Focus on performance measures, baseline performance data, common implementation problems, lots of SEI Technical Report references.

Daskalantonakis, Michael. "Achieving Higher SEI Levels," IEEE Software, 7/94, pp. 17-. Technique for performing incremental assessments (mini-assessments).

Krasner, Herb. "The Payoff for Software Process Improvement (SPI): What it is and How to Get It", Software Process Newsletter, vol. 1, no. 1 (9/94), pp. 3-8. Describes what constitutes the payoff for SPI, refers to previous published reports of the payoffs, case histories of NASA SEL, Loral Space Shuttle, and HP, and describes how to measure payoff in process quality, product quality, project predictability.

Herbsleb, J., D. Zubrow, J. Siegel, J. Rozum, A. Carleton. "Software Process Improvement: State of the Payoff", American Programmer, 9/94, pp. 2-12. Summary of some of the benefits from CMM-based SPI that companies have reported.

Pressman, Roger S. "Getting Smarter: A Key Component of Software Process Improvement," American Programmer, 9/94, pp. 29-35.

Mogilensky, Judah, and Betty L. Deimel. "Where Do People Fit in the CMM?", American Programmer, 9/94, pp.37-43.

McDowell, Mark A.. "From Crisis to Quality: Managing the Software Development Process," American Programmer, 9/94, pp.44-49.

Rubin, Howard A., Margaret Johnson, Ed Yourdon. "With the SEI as My Co-Pilot: Using Software Process 'Flight Simulation' to Predict the Impact of Improvements in Process Maturity," American Programmer, 9/94, pp.50-57.

Binder, Robert, "Software Process Improvement: A Case Study", Software Development, 1/94, p. 59.

McConnell, Steve. "From Anarchy to Optimizing," Software Development, vol. 1, no. 1 (July 1993). Overview of the CMM, with some benefits from Lockheed.

"Capability Maturity Model, Version 1.1," Mark C. Paulk, et al., IEEE Software, 7/93, p. 18.