Complexity, Organizations and Change

Elizabeth McMillan

• Publisher: Taylor & Francis
• Year: 2003
• Print ISBN: 9780415314473
• eBook ISBN: 9781134379859
• Edition: 1
• Topic: Business
• Subtopic: Management
• Index: Business

Complexity science has seriously challenged long-held views in the scientific community about how the world works. These ideas, particularly about the living world, also have radical and profound implications for organizations and society as a whole. Available in paperback for the first time, this insightful book describes and considers ideas from

1 Introduction

In my view, there is today a considerable amount of outdated and unhelpful thinking about the world of organizations. Too many current approaches to organizational change are drawn from a world view that is no longer consonant with the early twenty-first century. Traditional notions of organizations and how to manage them may have suited more stable times, but they do not offer effective solutions to organizations coping with the fast-flowing uncertainties of the modern world. Monolithic bureaucracies with their vast economies of scale seemed appropriate to the industrialized world of the early twentieth century, but they struggled to survive with the transformations brought about by modern technologies and globalization trends. Toby Tetenbaum (1998) lists six major characteristics which, for her, encapsulate these changes.

1. New technologies which have transformed communications, electronics, consumer markets and speeded up industries.
2. Globalization, which has resulted in a world that is evermore connected and interdependent as information, money and goods move around the planet.
3. Globalization and new technologies, which together have sharpened competition and precipitated the rise and fall of market leaders.
4. Change, which is now happening faster than ever before in our known history.
5. Speed – an ‘incredible increase in technological speed is matched in business (product life cycles are measured in months not years) and in people’s lives (most of us feel we are running as fast as we can merely to stay in place)’ (Tetenbaum 1998: 23).
6. Complexity and paradox which are increasing as a result of all these changes and are making more and more difficult demands on managers used to seeking certainties and ‘either/or’ type solutions in order to bring about the ideals of stability and order.

Thus, in recent decades, the landscape in which organizations exist has changed almost beyond recognition, however, I do not believe that mainstream organization theory and practice have kept pace with these developments. Traditional management literature still considers the world from a linear cause and effect perspective and advocates controlling and planning processes as the way to help an organization survive and succeed. These notions are based on ideas and approaches derived from a classical, traditional, scientific world view that was developed some 300 years ago. It is a view rooted in the science and philosophy of the seventeenth century entwined with an emerging scientific methodology. It is a view and a way of looking at the world that is so woven into the fabric of western thought and everyday life as to be indiscernible. It is the air that we breathe. This book argues that though the natural order still unfolds as it always did, the world we have created and where we live our everyday lives has significantly changed, but our thinking has yet to make the same kind of shift. Modern science has combined research and imaginations and made them fact, for example, in the shape of laser applications, multimedia communications, robotics, interplanetary vehicles, nanotechnology and a range of biotechnologies. I believe that it’s time that mainstream management and organization theory and practice caught up.

The prevalent literature on organizations and management has been challenged in recent years by a number of writers. People like Gareth Morgan and Charles Handy have broken with tradition and offer insightful and innovative ways of reconsidering life in organizations. Morgan in his book Images of Organization uses metaphor to explore our notions of organizations in an exciting and transforming way. Handy in The Age of Unreason and The Empty Raincoat offers new and challenging visions of the future of life and work. The development of ideas on learning and the learning organization, by people like Peter Senge, Margaret Dale, Mike Pedler, John Burgoyne and Tom Boydell, offers a demonstrably useful set of ideas for today’s organizations. Ricardo Semler, the Brazilian manager, describes in his book Maverick how he broke the established rules and approaches to management to forge a dynamic and egalitarian workplace. Thus the contemporary literature on managing organizations is being injected with some powerful and fresh ways of thinking.

Recently another set of management literature reminiscent of the more radical ideas on change has been emerging – the literature based on the new sciences. This has been informed by ideas from chaos theory, complexity science and research at the Santa Fe Institute in the USA and in European institutions. These ideas have been taken up by people who have a major interest in organizations and organizational change, people like Ralph Stacey, Margaret Wheatley, Ikujiro Nonaka, and Richard Pascale. I would argue that truly radical thinking on organizations is needed if we are to better understand the world in which we live and work and that drawing on these scientific understandings can help us to achieve this.

The emergence of complexity science as the last century drew to a close had profound implications for the scientific community and a world view based on Newtonian–Cartesian thinking. Complexity now challenges the predominant scientific tradition which many consider to be highly successful, and it is making a radical and significant contribution to new scientific thinking and understanding. It challenges long held notions of predictability and the importance of small events or minor differences. This has major implications for our notions of change, how it occurs, the types and degrees of change and thus for strategic planning and associated activities. Complexity science challenges the ubiquitous use of over-simplified, linear approaches to events and happenings that has led to the development of unreal thinking and unrealistic models of the world. Further, it restores the importance of emotion and intuition to their rightful place alongside rationality and logic and it takes a holistic perspective on a world that has both consciously and unconsciously become over reductionist and compartmentalized.

Paradigm shifts are, of course, very difficult to achieve as established paradigms tend to be strongly supported by those who have built their reputations or advanced their careers by embracing such a perspective. Further, they are usually deeply rooted in people’s thinking and in the way they unconsciously interpret the world around them. They are often accepted unquestioningly and are only seriously questioned by very few original thinkers. However, ideas and insights derived from complexity science have now spread across the western world and are being explored and applied in many non-scientific domains such as education, politics, philosophy, management, economics and social studies. A considerable number of writers, thinkers and pragmatists now consider that a new paradigm is coming into being – the complexity paradigm. It is a view which I support and which I shall explore further in this book.

But how can understanding of the complexity paradigm significantly and pro-foundly affect the way people think and act in society at large, and most specifically in organizations? Does it offer helpful new ways of interpreting and understanding the world of organizations? This book seeks to explain and describe how viewing the world from a complexity paradigm perspective can shed fresh light on a range of organizational problems and issues, and suggest innovative and ground-breaking ways of reshaping the organizational world so that it is more in tune with the times. It does so by seeking to build on the existing management literature base which draws on complexity science. Also it reviews some existing mainstream theories and practice and looks at complexity influenced activities in a number of organizations. It offers an in-depth case study of a traditional, complex and complicated organization that used complexity ideas to challenge and change the predominant culture, as a real life example of complexity ideas at work. Observations, models and some new knowledge derived from this detailed case study are offered in the belief that they may help organizations to cope more effectively with the waves of uncertainty and change that already are the hallmark of the twenty-first century.

In writing this book I have considered a sweep of ideas and insights based on an approach to the literature on complexity that is wide-ranging and eclectic, including writers from many backgrounds and disciplines. The case study I present investigates in considerable detail a change process in one organization, but overall I have attempted to provide an integrative and holistic approach. One that combines the fine detail that comes with close investigation with a focus that takes a more distant and all embracing perspective. Organizations play a very large role in both our working and non working lives and thus a key role in the overall dynamics of society and I have sought to acknowledge this in the approach I have taken in this book.

The new sciences encourage us to look not just for formal structures and content but for patterns, flows and processes. Thus in writing this book I have sought to add to the existing literature on complexity and management by focusing on the people dimension, that is the behaviours, emotions and learning patterns and processes that flow through an organization and give it its life and characteristics.

I have spoken broadly of complexity science and the complexity paradigm and they will be explored in more detail in other chapters of this book, but what is my overall view of this brave new science and why and how do I intend to discuss its many aspects? For the reader who is largely unfamiliar with complexity science territory it perhaps needs to be stated first of all that complexity science involves a broad spectrum of disciplines: biology, physics, mathematics, chemistry and meteorology. I also would include chaos theory and evolutionary biology as features of complexity science. Thus understanding the many facets of complexity science would appear to offer a serious challenge to those with limited scientific education. So why does this book offer a broad overview of the history and development of complexity science and describe at some length complexity theories and ideas? The answer is that, in my view, anyone interested in complexity science and its applications in management and organizations should understand the origins of the ideas they may be working with. Only by understanding the basic concepts on which applications and theories are derived will the intelligent reader be able to form their own sound judgements. Having the knowledge and understanding to critique ideas derived from complexity is, in my opinion, essential if one is not to be misinformed, mislead or even cheated by expensive consultants bearing gifts.

A major aim of this book is to build a bridge between the world of science and the world of organizations and management, and to contribute to the ongoing debate about complexity science and its applications. Thus I describe the new sciences in order to investigate their relevance outside the scientific arena. It is not my aim in this book to assume the role of the expert scientific researcher nor to critique any scientific theories. (Indeed, I do not consider myself qualified to do so.) The ideas and theories put forward are drawn from the mainstream of the new sciences, and overall, are well accepted within that realm. Thus the main tenets of complexity science are covered but the list is meant to be informative and illuminating rather than exhaustive. Inevitably some names may be missing.

As complexity science is multidisciplinary, to cover all the main theories and ideas presents a significant challenge. In order to address this I have chosen to refer to the work of a number of well-known scientists in different disciplines and also to scientific writers who are recognized for their work on interpreting the work of others researching in the field. For example, James Gleick’s (1993) book on chaos describes the history and development of chaos and discusses the work of major contributors such as Lorenz, Feigenbaum and Mandelbrot. This is now considered by many to be the best introduction to the origins and essence of chaos.

As scientific knowledge increased over the last decades, so more and more sub-disciplines emerged with a focus on their own highly specialized domain. Biology, for example, is one traditional discipline that now has many new branches or specialisms and each specialism tends to have its own particular way of framing and describing the world. So, if one is considering a particular life event or a scientific phenomenon, which interpretation is the right one? The answer is, of course, that there is no right interpretation and this is a very important message for anyone coming to understand the complexity paradigm. I shall illustrate this by describing a story that Steven Rose tells in his book Lifelines.

Five biologists are sitting by a pool enjoying a picnic when a frog that was sitting close by makes a sudden leap into the water. This prompts a discussion between the biologists on why the frog jumped. One of them, a physiologist, states that the creature jumped because the muscles in the frog’s leg are responding to signals from the brain, which is itself responding to a message from the frog’s retina when the snake was spotted. Rose describes this as a ‘simple “within level” causal chain’ whereby one event follows another ‘all within a few thousandths of a second’ Rose (1998: 10). But this explanation is challenged by another of the biologists, an ethnologist. The physiologist has explained how the frog jumped but not the reason why. The animal behaviourist explains that the creature jumped in order to avoid being eaten by the snake. It was a goal-directed action and can only be understood within its environmental and social contexts. Thus the ethnologist views the notion of cause quite differently from the physiologist.

The third biologist is a developmentalist and finds the other two explanations inadequate. From a developmental point of view the only reason why the frog can jump is because during its development from fertilized egg to adult frog its brain, nerves and muscles were wired up in such a way that such a sequence of activity was highly probable in the circumstances. As Rose (1998: 12) observes: ‘the onto-genetic approach introduces a historical element into the account: the individual history of the frog becomes the key to understanding its present behaviour’.

The fourth biologist, an evolutionist, however, is not satisfied by any of the explanations to date. The evolutionary explanation is that the frog jumped because ‘during its evolutionary history it was adaptive for its ancestors to do so at the sight of a snake; those ancestors that failed to do so were eaten, and hence their progeny failed to be selected’ (Rose 1998: 12).

Finally, the fifth biologist, a molecular biologist, speaks up and claims that they have all missed the point.

The frog jumps because of the biochemical properties of its muscles. The muscles are composed largely of two interdigitated filamentous proteins, called actin and myosin, and they contract because the protein filaments slide past one another. This behaviour of the actin and myosin is dependent on the amino acid composition of the two proteins, and hence on chemical properties, and hence on physical properties.

(Rose 1998: 13)

So here we have five different, yet valid explanations, from five different sub-disciplines of biology. So who do we believe? As Rose points out, we have to consider all of the explanations and possibly others too. There is no one simple answer and it is an approach which I have sought in writing this book. The one right answer or explanation is a way of thinking that belongs to the old scientific paradigm, yet one encounters it everywhere, even in writings on complexity science. It should also be borne in mind that simple causal methods of explanation tend to paint linear pictures. Rose reminds us that notions of ‘cause’ have been troublesome since the days of Aristotle, so it may be more advisable to think in terms of clear temporal sequences. Again I shall endeavour to avoid the pitfalls of cause and effect approaches, but I have to acknowledge that it is embedded well into my thinking as part of my traditional educational experiences. That is not to say that it is without value, but it tends to oversimplify the complexities of our world, and can encourage inappropriate, and sometimes over simplistic interpretations of complex issues.

Scientists have their own ‘languages’, for example, the language of the physicist is not always understandable by the chemist and vice versa. Further, as the above story illustrates there are specialist terms used by sub-disciplines within mainstream disciplines. Thus, as scientific research builds up ever more massive knowledge databases, so the many languages of science increase their vocabularies and sub-languages emerge along with new sub-disciplines. I have sought in this book to write in a way that is accessible to the non-scientific reader and to avoid a vocabulary that is highly scientific wherever possible. I have thought for some time that complexity science needs to develop an accessible language that is rooted in verifiable science, but is not itself exclusively scientific and dependent upon an esoteric scientific vocabulary to explain itself.

The new sciences challenge many aspects of the existing paradigm and in so doing indirectly offer a challenge to the way organizations are structured and behave. If complexity provides us with explanations about how the world works then, I would argue, these explanations should be of value in considering how organizations work and how to change them. I was able to test this hypothesis in a real life laboratory and the ideas and insights I present in this book draw heavily on empirical data from my research and a range of practical experiences.

This book aims to embrace many aspects of how organizations might be transformed by using ideas drawn from complexity science. Therefore it seeks to consider a spectrum of implications, whether they have been strategic, operational or employee based. But there is a special emphasis on considering the role that the people in an organization play in transforming that organization. The term ‘people’ refers to everyone who works in an organization regardless of role or status. I would contend that they, rather than high level, strategic planning processes and formal change interventions, especially top-down ones, are the real key to understanding organizational change. Thus this book argues that it is human dynamics which bring about real ‘second order’ change and renewal. That is change that is not surface but deep down, where thinking and behaviours become significantly different.

So far this chapter has sought to introduce the main underpinning themes of the book and the approaches I have taken in writing it. In the final section of this introduction I outline the contents on a chapter-by-chapter basis.

A complete chapter in this book, Chapter 2, is dedicated to the history and development of those ideas and theories, and those chiefly credited with their development, which have collectively become known as complexity science. Thus a significant part of this chapter includes the basic tenets of chaos theory and draws heavily on the writing of Gleick (1993). The following are included in this section: the butterfly effect (or sensitive dependence on initial conditions) and the work of Edward Lorenz; Benoit Mandelbrot and the development of fractal geometry; strange attractors; notions of order and disorder; the edge of chaos; concepts of universality; patterning and flow. The chapter then moves on to the emergence of complexity science with reference to the writings of Fritjof Capra; the work of Ilya Prigogine on dissipative structures and self-organization; John Holland on complex adaptive systems and emergence; and Stuart Kauffman and others on biology and fresh interpretations of evolution. This chapter also includes a brief discussion on the use of the terms ‘chaos’ in the ordinary vernacular.

In order to explore the importance of the ideas and theories arising from the new science it was important to consider the background against which they arose. Thus Chapter 3 describes the origins and development of the classical, scientific tradition founded by Galileo, Newton, Descartes and others. This established a mechanistic, linear view of the world which took a reductionist, cause and effect approach to science. Human beings were seen as machine-like and the intellect was highly valued and the senses disregarded. In spite of challenges from the Romantics in the early nineteenth century and from quantum physics and relativity theory in the twentieth century this world view continued to exert a very powerful influence on all aspects of life. Many early technologists and the founders of industrial society in the Western world looked to science to provide ideas and models as to how to establish their new industries. This chapter includes a review of mainstream organization theory and practice including: organizations as machines; bureaucracy; classical management theory; scientific management; the behavioural influences; and developments in the 1970s and 1980s. The end of the twentieth century saw organizations trying to cope with rapid change and uncertainty and experimenting with a host of ideas including downsizing and business process re-engineering, and the emergence of learning organization concepts. This chapter considers how much ideas derived from the classical, scientific view still exist in today’s organizations and refers to Handy (1990, 1993), Morgan (1986) and others to offer considerable evidence to support the view that the influence is still a powerful one. Also included is a discussion of the significance of language and metaphor because of the insights they offer us into organizational life.

Many organizations and their managers assume that change can be managed and controlled in an orderly fashion until the new desired state is arrived at. This is a mechanistic, linear approach to change which assumes that equilibrium can be achieved and maintained in a rapidly changing world and is typified by the use of deliberate strategies as described by Mintzberg and Waters (1989). Chapter 4 looks at how the way organizations think about change affects their ability to handle it. It includes a brief survey of how perceptions of change and the nature...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Illustrations
• Preface
• Acknowledgements
• 1 Introduction
• 2 Introducing complexity
• 3 Organizations and the long shadow of scientific tradition
• 4 Plus ça change … the more it stays the same?
• 5 Transforming organizations using complexity
• 6 Changing the Open University – a case study
• 7 Complexity in action
• 8 Future possibilities, future choices
• Appendix 1: Plans for change New
• Appendix 2: Diary of events
• Appendix 3: Conference issues
• Bibliography