A Guide to Selecting Software Measures and Metrics

Capers Jones

• Publisher: Auerbach Publications
• Year: 2017
• Print ISBN: 9781032476919
• eBook ISBN: 9781315314624
• Edition: 1
• Topic: Business
• Subtopic: Project Management
• Index: Business

Going where no book on software measurement and metrics has previously gone, this critique thoroughly examines a number of bad measurement practices, hazardous metrics, and huge gaps and omissions in the software literature that neglect important topics in measurement. The book covers the major gaps and omissions that need to be filled if data about software development is to be useful for comparisons or estimating future projects.

Among the more serious gaps are leaks in reporting about software development efforts that, if not corrected, can distort data and make benchmarks almost useless and possibly even harmful. One of the most common leaks is that of unpaid overtime. Software is a very labor-intensive occupation, and many practitioners work very long hours. However, few companies actually record unpaid overtime. This means that software effort is underreported by around 15%, which is too large a value to ignore. Other sources of leaks include the work of part-time specialists who come and go as needed. There are dozens of these specialists, and their combined effort can top 45% of total software effort on large projects.

The book helps software project managers and developers uncover errors in measurements so they can develop meaningful benchmarks to estimate software development efforts. It examines variations in a number of areas that include:

• Programming languages
• Development methodology
• Software reuse
• Functional and nonfunctional requirements
• Industry type
• Team size and experience

Filled with tables and charts, this book is a starting point for making measurements that reflect current software development practices and realities to arrive at meaningful benchmarks to guide successful software projects.

Chapter 1

Introduction

As the developer of a family of software cost-estimating tools, the author is often asked what seems to be a straightforward question: How accurate are the estimates compared to historical data?

The answer to this question is surprising. Usually the estimates from modern parametric estimation tools are far more accurate than the historical data used by clients for comparisons! This fact is surprising because much of what are called historical data are incomplete and omit most of the actual costs and work effort that were accrued.

In some cases historical data capture only 25% or less of the full amount of effort that was expended. Among the author’s IT clients, the average completeness of historical effort data is only about 37% of the true effort expended when calibrated by later team interviews that reconstruct the missing data elements such as unpaid overtime.

Quality data are incomplete too. Most companies do not even start measuring quality until after unit test, so all requirement and design defects are excluded, as are static analysis defects and unit test defects. The result is a defect count that understates the true numbers of bugs by more than 75%. In fact, some companies do not measure defects until after release of the software.

Thus when the outputs from an accurate parametric software cost-estimating tool such as Software Risk Master™ (SRM), COCOMO II, CostXpert, ExcelerPlan, KnowledgePlan, True-Price, SEER, or SLIM are compared to what are called historical data, the results tend to be alarming and are also confusing to clients and client executives.

The outputs from the estimating tools often indicate higher costs, more effort, and longer schedules than the historical d...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Preface
• Acknowledgments
• About the Author
• 1 Introduction
• 2 Variations in Software Activities by Type of Software
• 3 Variations in Software Development Activities by Type of Software
• 4 Variations in Occupation Groups, Staff Size, Team Experience
• 5 Variations due to Inaccurate Software Metrics That Distort Reality
• 6 Variations in Measuring Agile and CMMI Development
• 7 Variations among 60 Development Methodologies
• 8 Variations in Software Programming Languages
• 9 Variations in Software Reuse from 0% to 90%
• 10 Variations due to Project, Phase, and Activity Measurements
• 11 Variations in Burden Rates or Overhead Costs
• 12 Variations in Costs by Industry
• 13 Variations in Costs by Occupation Group
• 14 Variations in Work Habits and Unpaid Overtime
• 15 Variations in Functional and Nonfunctional Requirements
• 16 Variations in Software Quality Results
• 17 Variations in Pattern-Based Early Sizing
• 18 Gaps and Errors in When Projects Start. When Do They End?
• 19 Gaps and Errors in Measuring Software Quality Measuring the Cost of Quality
• 20 Gaps and Errors due to Multiple Metrics without Conversion Rules
• 21 Gaps and Errors in Tools, Methodologies, Languages
• Appendix 1: Alphabetical Discussion of Metrics and Measures
• Appendix 2: Twenty-Five Software Engineering Targets from 2016 through 2021
• Suggested Readings on Software Measures and Metric Issues
• Summary and Conclusions on Measures and Metrics
• Index