Texas National Energy Modeling Project

An Experience in Large-Scale Model Transfer and Evaluation

Milton L. Holloway

• Publisher: Academic Press
• Year: 2013
• Print ISBN: 9780123529503
• eBook ISBN: 9781483260907
• Topic: Business
• Subtopic: Real Estate
• Index: Business

Texas National Energy Modeling Project: An Experience in Large-Scale Model Transfer and Evaluation reports on the Texas National Energy Model Project (TNEMP) experience. The TNEP was tasked with providing an independent evaluation of the Energy Information Administration's (EIA) Midterm Energy Forecasting System. It also provided recommendations to the Texas Energy Advisory Council concerning the maintenance of a national modeling system by the Council to evaluate Texas impacts within a consistent national modeling framework. The book provides all of the summary material documenting the entire experience, sequentially, from beginning to end. It first lays out the purposes of TNEMP, the organizational structure for the study, and an explanation of the evaluation criteria used to guide the model critiques. It summarizes in some detail the important findings of each of the 11 studies contained in Part II published under a separate cover. It presents the National Advisory Board's assessment of the integrity of the evaluation project, their views of important outcomes of the TNEMP experience, and important recommendations to TNEMP and EIA. The final chapters contain an overview reply by EIA and a summary of a workshop held at the end of the project to discuss substantive issues raised by TNEMP.

CHAPTER 1

PROJECT PURPOSE AND STRUCTURE

Milton L. Holloway

Publisher Summary

Policymakers are using large-scale models as there is no alternative because of the complexity of potential impacts from policy decisions, but at the same time, they are skeptical of the reliability of forecasts and calculations from models. There is also great acceptance by the populace at large regarding the results coming from computer analyses that in their minds seem to represent the epitome of technological solutions to problems. Analyses are seen as more believable as they are based on computer technology. This chapter discusses the use of large-scale econometric models in the debate over national economic policy. The most prominent among these large-scale models used in economic policy planning by government include Data Resources, Inc. (DRI), Chase and Wharton modeling efforts. Apart from the use of econometric models in general economic planning by government, there has been a major surge of new computer modeling for purposes of energy policy planning and analysis. The disciplines of economics, operations research, statistics, urban and regional planning, and engineering are heavily involved in quantitative approaches to modeling systems of the real world. The chapter discusses the major task to improve the usefulness of models and the judgments of professionals involved in public policy analysis. The central issue is the procedures by which the reliability of large-scale models, especially those used in public policy work, can be established and made transparent to distinguish between the influence of professional, subjective judgments, and the influence of information that is reproducible by others. The Texas National Energy Modeling Project (TNEMP) has made some contribution to the goal of increased model credibility by transferring and operating Midterm Energy Forecasting System (MEFS).

INTRODUCTION

Large-scale models are a product of our times and seem to be growing in the importance of their use in government policy work. Policymakers are using large-scale models, having no alternative due to the complexity of potential impacts from policy decisions, but at the same time they are skeptical of the reliability of forecasts and calculations from models. There is also great acceptance by the populace at large of results coming from computer analyses which in their minds seem to represent the epitome of technological solutions to problems. Analyses are somehow seen as more believable if they are based on computer technology.

A recent article in Fortune characterizes the importance of this relatively new modeling phenomena in policy work.¹

“When the history of economic policymaking in the turbulent late 1970s is written, an important part of this story will be about the widening impact of econometric models on federal policy decisions. The wondrous computerized models—and the economists who run them—have become new rages on the Washington scene. These days it seems every spending and tax bill is played into mathematical simulations inside the computer. The model managers themselves are star witnesses before congressional committees whose members seek to define the future. And what these machines and their operators have to say has come to have a significant bearing on what Washington decides to do….

On more than one occasion the models have contradicted pronouncements by senior government figures. A small model at the Council of Economic Advisors, for instance, indicated that James Schlesinger, the Secretary of Energy, was talking through his hat last March when he predicted that disastrous consequences would flow from last winter’s coal strike.”

The examples go on and on of the recent use of large-scale econometric models in the debate over national economic policy. The most prominent among these large-scale models used in economic policy planning by government include DRI, Chase and Wharton modeling efforts.

Outside of the use of econometric models in general economic planning by government there has been a major surge of new computer modeling for purposes of energy policy planning and analysis. Some of the better known of these modeling efforts include the Department of Energy’s (DOE) Project Independence Evaluation System (PIES), now known as Midterm Energy Forecasting System (MEFS), the Brookhaven Energy Reference System, the Stanford University PILOT modeling effort, the Baughman-Joskow Regional Electricity Model, and many others.

Greenberger, et al, in a recent book² have classified the types of models by five categories related to disciplines which are listed as (1) linear economics, (2) operation’s research, (3) statistical economics, (4) urban and regional development, and (5) engineering. The nine specific methodologies listed within these disciplines include (1) input/output analysis, (2) linear programming, (3) two-person zero sum games, (4) probabilistic methods, (5) algebraic methods, (6) econometric modeling, (7) microanalysis, (8) land use analysis, and (9) systems dynamics. The disciplines of economics, operations research, statistics, urban and regional planning, and engineering are heavily involved in these quantitative approaches to modeling systems of the real world.

In energy policy analysis these models have been used most recently for projecting the world crude oil situation, the impact of U.S. Government conservation and production incentives on the U.S. economy, in producing annual outlook reports for the U.S. energy situation, and most heatedly in the national debate over natural gas pricing during the discussion of President Carter’s National Energy Plan (NEP).

The above quotations simply serve to point out the fact that large-scale models are very prominent in government policy planning work and are being used extensively—large-scale computer models are part of the professional and political life of today.

The current dilemma of policymakers in the use of large-scale models is further exemplified by a recent interview with Secretary Schlesinger:³

Q: The original NEP would have saved how many barrels of imports?

A: We estimated 4.5 million b/d.

Q: So the level would be about 7 million b/d in 1985?

A: Right.

Q: And this year’s legislation would save how much?

A: About 2.5 to 3 million b/d.

Q: So the level of imports in 1985 would be 9 or 10 million b/d?

A: Yes. Our estimate is in that range. Those models (which are used in forecasting) aren’t worth very much.

Q: I’m glad to hear you say that—after the bill is passed you admit it, is that it?

A: I stated it rather forcefully from the first. Matter of fact, my troops had to force me to include these estimates from the PIES model. What we are trying to do is to change behavior, change behavioral reactions. Yet we get our estimates of the future from a model that draws on parameters of past behavior. It’s just logically inconsistent.

Q: A model cannot predict the future—it can only estimate based on history?

A: Another thing wrong is that we should never have had only point estimates—we should have had a range. The volume of oil imports is contingent on such factors as nuclear power plant start-ups, the amount of natural gas produced or coal used, etc. I find it exceedingly difficult to predict how much natural gas we will produce in 1985. The resource base there is much more substantial than the oil base. How much really depends partly on whether intrastate producers sulk, believe in the “regulatory nightmare” myth, or whether they will just get cracking.

Q: Do you think the resource base on gas is still pretty large? Potentially undiscovered? You feel better now than a year ago?

A: Just look at the results. I’m not sure the numbers are different. I feel a little better about gas.

Q: You think the higher price of gas will encourage more exploration? More development?

A: We’re estimating production of 2 trillion cubic feet more gas by 1985 in the lower 48 states under this legislation, compared with the status quo.”

The Senate Energy Committee recently asked Dr. Herman T. Franssen, formerly with the Congressional Research Service, to look into the subject of forecasts of energy supply and demand, both national and international. Dr. Franssen’s main conclusion is contained in the summary of his report.⁴

The main reason for inaccuracies is that forecasts are influenced by the Zeitgeist—the spirit of the times. When the spirit is optimistic, the energy forecasts are optimistic. When all is gloom and doom, then everyone says we are running out of energy. Mathematical models, which are supposed to screen out such subjective influences, have been even less accurate than the subjective forecasts. As for the conflicting forecasts being made now with an eye towards 1990, one is no more likely to be accurate than another. Due to physical and human phenomena subject to surprise and frequent change, it is understandable that different projections of energy demand, supply, and prices can be believable at the same time. Finally, a lot depends on who hires the forecaster. The policymaker is no doubt aware that energy forecasters work for different clients, and that, for example, oil companies plan for future opportunities while governments have to plan for national supply security.

Seen from a slightly different perspective, Greenberger, et al, speak of the difficulty of modeling systems which include sociological behavior of people.⁵

Theory as a basis for a formal model is of greatest value when it refers to a reference system that changes only slowly. Unfortunately, policy areas are among the most volatile fields of application for models. Economist Robert M. Solow of MIT puts it this way: “One advantage the physicist has over the economist is that the velocity of light has not changed over the past thousand years, while what was in the 1950s and 1960s a good wage and price equation is no longer so.”

Theories that change only slowly over time may not keep up with policy areas that change rapidly and consistently. Recognizing this shortcoming in theory a modeler may personally adjust the projections of his model to reflect an impending strike, war, bankruptcy or other events and trends not covered by the theory. In a model of the national economy “add factors” may be applied to the modeler’s projections to incorporate the modeler’s judgment and intuition about economic trends. The practice is widespread. Many well-known econometric models use some form of judgmental adjustment. Louis Mumford has observed recently that computer models can acquire a God-like authority and the results can be taken as gospel. A model masquerading as an oracle may be nothing more than an advocate in technological guise.

However, policymakers must use large-scale models despite a scepticism of their predictive reliability, because they have no alternative way to handle these complexities. Secretary Schlesinger’s interview exhibits this dilemma on the part of policymakers.

The major task that lies before us is to improve the usefulness of models and the judgments of professionals involved in public policy analysis. The central issue is the procedures by which the reliability of large-scale models, especially those used in public policy work, can be established and made transparent—to distinguish between the influence of professional, subjective judgments and the influence of information that is reproducible by others. The Texas National Energy Modeling Project (TNEMP) has made some contribution to the goal of increased model credibility by transferring and operating MEFS.

PROJECT PURPOSE

The first purpose of TNEMP is to provide an independent evaluation of the Energy Information Administration’s (EIA) MEFS. The evaluation will provide guidance to users of MEFS concerning the level of confidence one may have in the results of the models for government energy policy analysis purposes. The evaluatio...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright
• CONTRIBUTORS
• PREFACE
• READER’S GUIDE AND ACKNOWLEDGEMENTS
• LIST OF ACRONYMS
• HIGHLIGHTS
• CHAPTER 1: PROJECT PURPOSE AND STRUCTURE
• CHAPTER 2: ANALYSIS TEAM CONCLUSIONS AND RECOMMENDATIONS
• CHAPTER 3: FURTHER MODELING ASSESSMENT AND DEVELOPMENT FOR TEXAS
• CHAPTER 4: NATIONAL ADVISORY BOARD REPORT
• CHAPTER 5: DEPARTMENT OF ENERGY REVIEW COMMENTS
• CHAPTER 6: WORKSHOP ON SUBSTANTIVE MODELING ISSUES: WASHINGTON, D.C. 27 August 1979
• APPENDIX A: DOCUMENTATION OF USES OF MEFS
• APPENDIX B: EVALUATION OF DOE/MEFS TRANSFERABILITY AND DOCUMENTATION
• EFFECTS OF RESOURCE ESTIMATES AND RATES OF FINDING ON PROJECTION OF CRUDE OIL AND NATURAL GAS PRODUCTION
• THE INVESTMENT PROCESS IN THE DOE/MEFS OIL AND GAS SUPPLY MODELS: ANALYSIS AND IMPLICATIONS
• CRUDE OIL PRODUCTION FROM STRIPPER WELLS
• EVALUATION OF DOE/MEFS OIL AND GAS SUPPLY MODELS: BEHAVIOR OF THE COMPUTER MODEL
• EVALUATION OF THE DOE/MEFS COAL SUPPLY MODEL: BEHAVIOR OF THE COMPUTER MODEL
• EVALUATION OF STRUCTURE AND ASSUMPTIONS OF THE DOE/MEFS PETROLEUM REFINERY AND SYNTHETIC FUELS MODELS
• THE REPRESENTATION OF THE ELECTRIC UTILITY SECTOR IN THE DOE/MEFS: OVERVIEW AND COMMENT
• THE TRANSPORTATION MODEL IN THE DOE/MEFS: DOCUMENTATION AND CRITIQUE
• EVALUATION OF DOE/MEFS DEMAND MODEL
• MACROECONOMIC/MICROECONOMIC INTERFACE
• INDEX