1
Why Bureaucracy Fails
This chapter began as a talk to an audience of military and nonmilitary personnel from the Department of Defense, and some invited academics. The topic intially was military bureaucracies and the possibility of making them into innovative institutions. This was one in a series of remarkable seminars organized over the years by Andrew Marshall, the director of net assessment for the DOD, who never asked for anything but a freewheeling intellectual discussion from us. But the military men were truly interested, being dedicated bureaucrats constantly, and often dangerously, threatened by their own bureaucracies. They perhaps better than anyone understand the fatal weakness, and yet the indispensibility, of such institutions, a situation that is our contemporary Catch 22. And so they were properly fascinated with the idea that the drive to make their bureaucracies more efficient mightâindeed mustâbackfire. Death from friendly fire is something they know all about.
We are all either bureaucrats now or so enmeshed in bureaucracies that we might as well be. I have just completed some eighteen hours of work acting as an unpaid accountant for the federal and state governments in the preparation of my income tax returns; returns based on records accumulated with many other hours of effort over the tax year. This I have in common with us allâhonorary government bureaucrats. I like to think that as a university teacher and scholar I am not really a bureaucrat, but really I am. I hold an office by appointment and merit, not by hereditary title or family connections. I hold it because I fulfill certain conditions and meet certain qualifications. When I leave it it will be filled by someone else with the appropriate qualifications. I receive a regular salary, not payment in kind or occasional payments, and have a career structure through which I can rise, achieving tenure for life and eventually a pension. I have to meet a host of bureaucratic requirements to keep this position, and I stand somewhere (although often it is not clear just where) in a hierarchy of decision makers in my institution. The institution itself is a complex of hierarchies of office holders like myself, most of them as baffled by it as I am. Someone once said of the University of London (to which I also once belonged) that an intelligent man with a certain amount of time and application should be able to understand how it worked, but that no intelligent man would dream of trying. However (the university bureaucratâs favorite word) we must try, if only because, by a weird recursion, it is part of the bureaucratic job description of a social scientist that he try to understand such things. But as my title suggests I intend to beg the question, since most studies of bureaucracy are premised on the assumption that they should indeed study âhow it worksâ when it is blatantly obvious to those of us who belong to them or tangle with them that they do not work. They are indeed self-perpetuating and can bumble along for quite some time, and we must examine why. But they do not work in the sense of fulfilling their purposes, aims, or goals, which are, after all, usually quite rationally and explicitly set out in their charters. Even if in their own terms they achieve âefficiencyâ (another favorite bureaucratic word)âthat is, they rationally adapt means to endsâthey do this at the expense of those they are supposed to serve and those who serve them. Show me a happy bureaucrat and Iâll show you a warped human being. Oh, they exist all right, and that is another part of the problem.
Bureaucracies fail because of what Marx might have called their âinternal contradictions.â This is not news to social scientists, I suppose. What may be news is that these contradictions are inevitable. Behind most social science attempts to analyze the failures of bureaucracy lies the implicit assumption that with enough rational and scientific enquiry we will be able to pinpoint the trouble and cure it. Indeed, if there were not such an assumption no one would be willing to fund the research in the first place. It is assumed, not unreasonably, by the world at large, that it is the business of social science to come up with ways to patch and mend our torn social fabric. They do not pay us to say âwhatever you do wonât workâ anymore than they pay medical researchers to say âthere is absolutely no hope ever of curing this disease, sorry about that.â Again, the funds would dry up. So the social scientists plug away with studies of conflict resolution, management style, recruitment strategies, motivational problems, work satisfaction, informal organization, information flow, and all the familiar list. A lot of this is quite interesting from my point of view since it helps to show why bureaucracies can never work, but my colleagues are uniformly horrified when I offer such a justification for their efforts. They are infected with the basic rational utopianism of the social science agenda: give us enough time and enough grants and we will cure this chronic case of social malfunction. For one thing is obvious to both of us: there is no way that we can do without bureaucracies; modern society is utterly dependent on them for its existence. To say that they are hopelessly flawed by their very natures is like saying that modern medicine simply cannot cure the diseases we fear most (as is indeed the case, but we wonât go into that one.) The thought is unbearable. Something we are utterly dependent on simply must be made workable. It is that simple. The alternative is a kind of death sentence, or at least a sentence to a life of almost unendurable pain.
Fortunately no one is giving me a grant to do this research, which is not really research anyway but simply informed observation. So having no utopian agenda, and no masters to please, I can tell it like it is. The âinternal contradictionsâ of bureaucracy are indeed chronic; they are incurable. No amount of social science tinkering is going to do more than cover the patient with bandaids. But like the Fisher King with his suppurating wound, the patient will not easily die; it may equally suffer eternally unless there is some catastrophic intervention. We are dealing then with a chronic invalid who is absolutely necessary to our continued existence in society as we know it.
Despite all the social and psychological studies then, why do we not seem able to determine why bureaucracy fails as a matter of principle! The short answer is the time-frame in which we pose the question. If we do not look at this, or any other institutional problem, in an evolutionary time-frame we are bound only to get partial answers. Most sociologists of bureaucracy are unwilling to look even at a deep historical time-frame, much less an evolutionary one. It was not always so. One of the great fathers of sociology, Herbert Spencer, who invented the idea of looking at social facts in an evolutionary time-frameâquite independently of Darwin in factâgives a brilliant account of how state bureaucracies emerged from the organization of royal households. The titles of various officers in the English government still reflect this origin: Lord Privy Seal; Lord Chamberlain; First Lord of the Treasury; Chancellor of the Exchequer. All these are âministersâ because originally they were royal servants: they âministeredâ to the king and met in his âcabinetâ or antechamber. But while this gives us some sense of historical depth, we have to plunge much deeper into history to get an answer to our question. We have to plunge in fact into that period of history when serious genetic changes were still taking place and our present anatomy and mentality was being formed. Normally we do not think of this as âhistoryâ and even condemn it officially to being âprehistoryâ and this is the origin of our error. Evolution is simply history over long periods of time during which genetic change and adaptation occurred. (These can occur over relatively short periods of time as well, but usually such changes are minor, as with, for example, the evolution of the overbite in modern man.) If we think in these terms then we have to understand the history of bureaucracy as encompassing a huge period of time when it did not exist. The fact of that nonexistence is crucial. During that period of several million years we were formed as a species. Bureaucratic institutions were only invented a few thousand years ago, and only became totally pervasive in recent times. We have then to ask the crucial question whether or not they are compatible with the kind of creature that we evolved to be.
This is not such a strange question once one gets used to it. It is no different in principle from asking whether one should keep animals in cages and train them to do tricks for our amusement. There is no question that this can be done, but is it good for the animals? Equally obviously we can invent vast bureaucratic structures and train people to live in them, but is it good for the people? The question is only strange to those who have been reared in a cultural orthodoxy that says that any animal, including the human animal, can be trained to do anything so no problem exists except the problem of adequate training. To such a mentality, if bureaucracy keeps failing it can only be because we are failing in our training methods, and the answer is to find out why and improve them. The DSMIII, the official diagnostic manual of the psychiatric profession, used to list as one of its accepted âemotional illnessesâ (those that qualified for medical insurance), âjob dissatisfaction,â thus reflecting exactly this thinking. People who hated their dull, repetitive, inhuman work, were âsufferingâ from âemotional illness.â The answer was to change their attitudes until they accepted their lot. You can look this up; I didnât invent it. I donât know if this is still listed, but it used to be. Needless to say, my approach is to treat this as problematic rather than settled. If people in general are persistently unhappy with something despite all efforts to improve their attitudes, then I suggest we at least look at the possibility that the thing in question might be inimical to their basic needs, and their misery might not stem from their personal inadequacies. If caged animals sometimes lash out and attack their trainers, should we put the blame on the animals or on the cages and the trainers where the animals were not supposed to be in the first place?
What then I am going to argue, and what Tiger and I stated baldly in The Imperial Animal in 1971, is that bureaucracies fail because they are, in some sense, inhuman. In some sense they are human because they are human inventions. But it is one of the paradoxes of an animal endowed with intelligence, foresight, and language, that it can become its own animal trainer: it can invent conditions for itself that it cannot then handle because it was not evolved to handle them. It has always been my point, when faced with any so-called breakdowns in modern society, to ask whether or not these are really breakdowns or whether or not they are in fact endogenous healing processes. The idea that they are breakdowns (as with teen gangs, teen pregnancies, strikes and industrial unrest, high divorce rates, illiteracy, declining educational standards, collapse of nuclear family values, terrorism, and the like) has to assume that they were somehow in a ânormalâ state to start with. This again begs the question. We do not ever really ask what the normal state of these institutions is. Indeed the social sciences do not know how to ask it. The normal state for them is the state we have created. It can be no other. It is again as if we are assuming that the cages and trainers are ânormalâ for the animals, so that their rebellions against this state are a kind of pathology.
For Max Weber, the great analyst of bureaucratic systems, it was modern capitalism and mass democracy that brought to fruition the âpureâ form of the institution. Historically it had existed in various proto-forms, but only in industrial society were its full virtues developed and realized. This was part and parcel of the march of society towards a rational form of organization, away from the traditional, sacerdotal, and hereditary forms of the preindustrial age. Hear Weber: âThe decisive reason for the advance of bureaucratic organization has always been its pure technical superiority over any other form of organization. The fully developed bureaucratic mechanism compares with other organizations exactly as does the machine with the non-mechanical modes of production.â Exactly. He lists the virtues succinctly: âPrecision, speed, unambiguity, knowledge of the files, continuity, discretion, unity, strict subordination, reduction of friction and of material and personal costsâŚâ It is interesting that in stressing these virtues perhaps he gets to the heart of the problem without realizing it. Thus, he stresses, that when fully developed, âIts specific natureâŚdevelops the more perfectly the more the bureaucracy is âdehumnanised,â the more completely it succeeds in eliminating from official business love, hatred, and all purely personal, irrational, and emotional elements which escape calculation. This is the specific nature of bureaucracy and it is appraised as its special virtue.â (I left outâmarked by the ellipsesâthe words âwhich is welcomed by capitalismâ since later experience has shown us that it is welcomed even more by socialism, even if it was a capitalistic invention.) It is indeed. But as we can see from the list, the things it eliminates are absolute necessesities of human existence even if they are irrational and beyond calculation. He continues: âThe more complicated and specialized the modern culture becomes, the more its external supporting apparatus demands the personally detached and strictly âobjectiveâ expert, in lieu of the master of older social structures, who was moved by personal sympathy and favor, by grace and gratitude.â In terms of the criteria of rationality and efficiency, these messy personalistic values of traditional society must succumb to the impersonal and universalistic virtues of bureaucracy. But Weber is concerned with what is right with bureaucracyâin the sense of what âfitsâ with modern society and its organizational needs. For those of us concerned with what is wrong with it, Weberâs list of what has to be abandoned again reads like a nostalgic reminder of some of the things that make life worthwhile: âsympathy and favor, grace and gratitude.â
We have to find somewhere to start in spelling out the basic flaws and their relation to the evolutionary context. So let us look at what I consider to be the main flaw: bureaucracy is a rational relationship between means and ends, but the means of the bureaucracy tend to become the ends of the bureaucrats to the point where the initial purpose of the enterprise gets totally lost. The examples are too numerous to cite and too well known to need elaboration. We all of us are either constantly faced with the frustrations this engenders, or are guilty of perpetuating this very behavior. We all have experienced hospitals, which ostensibly exist for the cure of the sick, where the preservation of the hospital hierarchy and the observation of its rules often take absolute precedence over the needs of patients. We are so disturbed by this obvious truth, since our lives and well-being are seriously at stake, that we need a whole industry of T.V. fantasy shows full of caring doctors and personally involved nurses and the like to bolster our morale and calm our worst fears. We are numb by now with examples from the world of crime where the police supposedly exist to see that justice is done but in fact are obsessed with âmaking convictions stickâ at all costs including perjury and planted evidence. Without the convictions there cannot be promotions and pensions. University departments and university administrations, need I remind my most obvious audience, are riven with this problem. The purposes of universities might be a bit vague, but âteaching, research, and serviceâ are usually high on the list of objective goals. But as we all know these ends become easily subservient to the means of promotion and control. University administrations are less interested in any of these than in expanding the control of the administration over the faculty. They will support research, for example, insofar as it makes them look good, not as an end in itself, and they will readily sacrifice teaching if there is a greater payoff for them in some other direction. Above all they seek a ceaseless expansion of the administration itself, and the patronage that goes with it.
The history of warfare is replete with examples of the means-ends confusion. One only has, to this audience, to mention âinter-service rivalryâ to touch off a chain reaction of examples. Here, maintaining the distinctiveness of different units becomes more important than winning the wars (the ostensible purpose again of a military organization.) In fact, if we are to believe some of the things we hear round this table, winning the wars is merely incidental to maintaining the special interests of the various services and units within the services. Huge sums of taxpayersâ money are yearly wasted on projects concerned with this maintenence of differential status. In that ultimate example of bureaucracy-for-its-own-sake, the United Nations, practically no one can tell you the provisions of the U.N. Charter. But each unit and sub-unit is fiercely aware of the competition for funds under the yearly budget scramble. All other functions are subordinated to the task of maintaining and increasing that share. This reaches its peak in the months before the budget presentations are due. Anyone who has dealt with this monster knows that it is impossible to get business doneâthe business for which the departments existâduring this part of the year. You will be told quite frankly, âNo one is available since they are all in budget meetings.â
As I said, we could produce endless examples, but you can do this exercise for yourselves. Or go down to the local motor vehicles office and try to get the simplest thing accomplished. But this is only one aspect of this particular flawâthe substitution of bureaucratic means for the ostensible ends of the organization. It is itself a result of a perhaps deeper flaw. All bureaucrats are obsessed with rules, and with the observation and enforcement of rules as an end in itself. They are rarely concerned with the purpose of the rules; it is enough that the rules exist and must be enforced. But people at large are not all that interested in rules except as means. People have goals and they employ strategies to achieve them. Rules are for them simply one set of aids or hindrances in the achievement of the goals. But for the bureaucrat (in his role as bureaucrat) the observance of the rules is the only goal. He is not interested in the personal goals and strategies of the ordinary citizen, but in that citizenâs observation of the relevant rules. The possibilities then for a head-on clash and a tangle of misunderstandings are multitudinous, and again the examples are too numerous and even banal to need reciting. âWe have our rules you know.â âYou canât have read the rules properly.â âIf only youâd followed the proper procedures.â âThere is a rule you know.â It is all too dismally familiar. And this underlines the point that what I am talking about here is not some esoteric theoretical deduction from evolutionary or organizational or psychological theory, but the kind of observations obvious to our common sense. What is not obvious to our common sense is how things got into this state, and here we must make a brief detour through our evolutionary history, because if we do not then we are still left with the standard social science answer that what is wrong must lie in a failure of our socialization or training or adjustment, and that it can in principle be put right. George Orwell and Franz Kafka knew that it couldnât be put right, but they could only describe how it might achieve its ultimate awfulness. We will try to underpin their humanistic insights with a little dash of science.
We have heard bureaucracies described as âcomplex adaptive systemsâ by the complexity theorists. This is a happy turn of phrase because it allows me to introduce the concept of âcomplex mal-adaptive systems.â We are not adapted to them, nor they to us. What can this mean? Adaptation, in the evolutionary sense, refers to the process whereby organisms become fitted to their environments. No organism can be âadaptedâ in an abstract sense; it is adapted to an environment. This is why the nature/nurture debate was always a silly one. The adaptations of any organism assume a specific environmental input. This is why I have defined âinstinctsâ as âan organismâs specific demands for specific environmental input.â But here is the rub: environments can change, or in the human case can be changed, faster than the organismâs genetic adaptations can keep up. Every organism has what evolutionary biologists call its âenvironment of evolutionary adaptednessâ (EEA). This is the environment in which its distinctive characteristics as a species evolved, and hence the environment that it âexpects.â We must never lose sight of this: adaptation is a relationship between organism and environment. What is âin the genesâ is there because of adpatations to this environment and âassumesâ the input of this environment for the realization of its behavior over the life cycle. To use a crude example, it does not much matter what is in the genes of a fish if the fish is not in water. Adaptation has produced genes that âassumeâ the input of water (often specifically either fresh or salt water); a fish out of water, that is, an orgamism out of its environment of evolutionary adaptation, is either dead or severly disabled. It might survive if the novel environment can mimic enough of the EEA to fool the genes, as it were. But how long or how well it can survive is a moot point.
The EEA of Homo sapiens sapiens is variously described as the late Pleistocene, or the upper Palaeolithic; in laymanâs terms the old stone age. We entered this little differentiated from our ape cousins except for our upright stance and small teeth, and we ended it as modern humans, our brains having grown three times in size over less than two million years. This is the context of our history of adpatationsâphysical, emotional, cognitive, and social. The brain is the major organ of this adaptation, although it followed on such features as the upright stance and the opposable thumb.
It is often argued that one of the things we developed in the period was also âgeneral intelligenceâ or âconsciousnessâ or something that in essence frees us from any kind of obligations to specific adaptations. Apart, however, from the great difficulty of defining or locating these features, even their most avid proponents have to admit to a substrate of domain specific modules (or algorithms), which are cognitive and emotional adaptations of a specific and not a general purpose kind. It could be that so-called general purpose intelligence (or consciousness) only appeared after language itself appeared, and represents the fact that we can talk about our own motivations and needs in a way denied other animals. âGeneral purpose intelligenceâ is a problem as a total explanation of what motivates us since it clearly provides no motivations, although it may help us to be endlessly malleable in trying to adjust our motivations to our environments. In the end we must ...