Cost-Effectiveness Analysis

A Systems Engineering Perspective

Howard Eisner

• Publisher: CRC Press
• Year: 2021
• ISBN:
  9781000532784
• Edition: 1
• Topic: Tecnologia e ingegneria
• Subtopic: Ricerca e competenze ingegneristiche e tecnologiche

This book provides an overview of cost-effectiveness analysis, which is a well-known and intuitive method for defining and choosing among a set of alternatives. This book relates cost-effectiveness analysis to systems engineering to solve everyday problems at home and the office. It can also be used in technical processes, system design, and project management.

Cost-Effectiveness Analysis: A Systems Engineering Perspective starts with providing an overview and background of cost-effectiveness analysis and how it's used. It then goes on to discuss cost-effectiveness concerning systems engineering and links its use to resolving military issues and problems. The book comes to an end with exploring the usage related to systems architecting, re-engineering office systems, and comparing its use to everyday life decision-making scenarios.

Targeted market includes general engineers, systems engineers, process engineers, project management, scientists, technologists, mathematicians, and lawyers.

1 WSEIAC, the Whiz Kids, and RAND

DOI: 10.1201/9781003196174-1

In the summer of 2019, an economist by the name of Alain Enthoven authored a paper [1] on Systems Analysis (SA) and the cost and effectiveness of weapon systems. He noted that Charles Hitch, Chief Economist of the RAND Corporation, was appointed by Robert McNamara, Secretary of Defense, as Comptroller of the DoD and that 1961 approximately the year of the SA thrust in his office. McNamara also called this thrust P, A, and E (Planning, Assessment, and Evaluation). Its main purpose was to evaluate DoD systems in terms of their value to the DoD, and make PPBS (Planning, Programming, and Budgeting) decisions as to which systems are needed to be built and which are not.

The Enthoven paper pointed out that, by 1965, three milestones occurred:

1. Hitch became the president of the University of California, Berkeley.
2. Enthoven was appointed Assistant Secretary of Defense for Systems Analysis (SA).
3. The DoD published their WSEIAC report [2], a definitive treatise on how the DoD would deal with weapon systems and their effectiveness. The key summary report defined weapon system effectiveness as the product of A, D, and C, where:
   • A = Availability
   • D = Dependability
   • C = Capability

These important steps set the stage for the calculation of the “effectiveness” part of “cost-effectiveness” analysis.

At approximately that time (1965–66), this author found himself in front of Alain Enthoven, exploring the possibility of moving from a small think-tank company in Silver Spring, Maryland (Operations Research Inc.), to Enthoven’s staff as part of the “Whiz Kids” in McNamara’s P, A, and E. The discussion was quite exciting but ultimately did not lead to a change in positions into the SA part of the DoD. I don’t recall the details of that conversation, but at this time I would call it at least a mismatch of needs and responsibilities associated with this author’s career and work interests.

In this reporting, this author is using “Whiz Kids” as the generic name of all the folks that inhabited the P, A, and E portion of McNamara’s DoD responsibilities. It was indeed a very key portion in terms of the analyses and evaluations that were being carried out there. And the WSEIAC report played an important role at that time in setting the stage for the effectiveness assessments of military systems.

The WSEIAC Analysis and Report

This report had several special parts:

• The principal findings, conclusions, and recommendations of the five WSEIAC Task Groups
• A 15-step procedure for cost-effectiveness assurance
• A section of the integrated summary that contains abstracts and summaries of each of the ten reports submitted by the five Task Groups
• Appendix I, which has a more detailed treatment of each of the ten report tasks
• Appendix II, which contains an application of the methodology, and
• Appendix III, which is a glossary of cost and effectiveness terms

To continue on with the top-level definition:

Effectiveness is defined as the product of Availability, Dependability and Capability, where

“Availability is a measure of the condition of the system, at the start of a mission, when the mission is called for at an unknown (random) point in time,”

“Dependability is a measure of the system condition during the performance a the mission, given that it is available at the start of the mission, and”

“Capability is a measure of the results of the mission, given the condition of the system during the mission (dependability)”

“Cost-effectiveness is the value received (effectiveness) for the resources expended (cost)”

A concise statement of the perceived problem:

a minimum acceptable requirement of a recent SOR is given piecemeal in terms of separate probabilities and performance limits without obvious relation to one another. When combined into an overall effectiveness number (using WSEIAC notions) these requirements suggest that if this system works less than 4 times out of 100, it is still acceptable. [2, page 5]

The overall Committee for the WSEIAC was composed of five task groups of about ten members each. The objectives of each Task Group are delineated as follows:

Task Group I

1. Review present procedures for effectiveness requirements.
2. Recommend a method for determining system effectiveness requirements that are mission responsive.

Task Group II

1. Review existing documentation of system effectiveness.
2. Recommend methods and procedures for measurement and prediction of system effectiveness in all phases of a system’s life.

Task Group III

1. Review current Air Force data collection and reporting systems.
2. Recommend uniform procedures for periodic status reporting to assist all management levels.

Task Group IV

1. Develop a set of basic instructions and procedures for conducting analysis for system optimization, considering:
   • Effectiveness
   • Cost
   • Program time scale
   • Refine current cost-effectiveness analysis techniques

Task Group V

1. Develop a management system designed to absorb and apply systems effectiveness experience retention.

Technical Reports

• AFSC – TR – 65 – 1: Final Report of Task Group 1 “Requirements – Methodology”
• AFSC – TR – 65 – 2: Final Report of Task Group II: “Prediction – Measurement”
• AFSC – TR – 65 – 3: Final Report of Task Group III: “Data Collection and Management Reports”
• AfSC – TR – 65 - 4: Final Report of Task Group IV: “Cost-Effectiveness Optimization”
• AFSC – TR – 65 – 5: Final Report of Task Group V: “Management Systems”

The top-level report is provided in two volumes:

Volume I – General Summary

Volume II – Integrated Summary

So for the timeline of this author, WSEIAC plays an important role, especially when observed from the perspective of a starting point for cost-effectiveness analysis. After all, one might argue that there is no better place to start from other than the treatise that defines effectiveness. So, we move forward from there to the WSEIAC recommendations, which in effect is the last statement of that AFSC report, back in 1965.

WSEIAC Recommendations

These are the essential recommendations of this significant report from the Air Force.

1. The basic analytical framework and effectiveness concepts should be accepted as an initial starting base.
2. The proposed effectiveness notions and evaluation techniques need to be validated on one or more Air Force systems. Changes in the techniques should be based upon the results of that validation. Contractual commitment needs to be, likewise, the validation results.
3. Procedures for test and demonstration of effectiveness, similar to those for reliability, should be initiated. (Another good starting point.)
4. The matter of “confidence” in the effectiveness results need to be further studied.
5. Specific impacts on incentive fee contracts need to be explored as soon as possible.
6. The preparation and adoption of standard terminology for systems effectiveness needs to be accomplished as soon as possible.
7. More attention needs to be paid to data acquisition (e.g., maintenance data).
8. The fine art of parameter estimation needs further definition and standardization.

More about the Whiz Kids

McNamara’s tenure as Secretary of Defense under Kennedy and Johnson was notable, as he established the Whiz Kids in the DoD. Stories of this period of time are mixed, however. On one hand, this collection of people was outstanding, and there was a set of new ways to address military system and force levels. This was the new era of quantitative analysis, with new respect for and knee-deep systems analysis, econometrics, cost-effectiveness analysis, and PPBS methodologies.

The negative side of the activity, however, was acknowledged to becoming mired down “in the fog of war.” McNamara and his people lost sight of the need to bring the war to a halt, responding to the outcries of the public at large. So, what might a bottom line be for that time and the contributions made or not made by this special secretary of defense?

An article in the Harvard Business Review summarizes by declaring that McNamara’s time represented the “evolution of Modern Management” [3]. There was general acceptance of quantitative analysis leading to the creation of increased shareholder value for the enterprise. But it was also true that during that time there was a new focus on contributions to society and on figuring out and implementing programs for the common good. Despite McNamara’s being the “architect” of a futile war, he was also president of the World Bank for some 13 years. So he was able to “change his tun,” appropriately, during different phases of his life and position. He brought rigor to the statistical analysis of the war effort, and then brought that same rigor to trying to “make the world a better place.” A Business Week article used the following phrase to characterize McNamara and his style of leadership.

he was a prize specimen of a remarkable breed in U.S. Industry – the trained specialist in the science of business management who is also a generalist moving easily from one technical area to another.

A Barry Goldwater quote was sharper: “McNamara was one of the best ...