Complexity Theory and Project Management

Wanda Curlee, Robert L. Gordon

• Publisher: Wiley
• Year: 2010
• Print ISBN: 9780470545966
• eBook ISBN: 9780470769744
• Edition: 1
• Topic: Technology & Engineering
• Subtopic: Industrial Engineering
• Index: Technology & Engineering

An insightful view on how to use the power of complexity theory to manage projects more successfully

Current management practices require adherence to rigid, global responses unsuitable for addressing the changing needs of most projects. Complexity Theory and Project Management shifts this paradigm to create opportunities for expanding the decision-making process in ways that promote flexibility—and increase effectiveness. It informs readers on the managerial challenges of juggling project requirements, and offers them a clear roadmap on how to revise perspectives and reassess priorities to excel despite having an unpredictable workflow. One of the first books covering the subject of complexity theory for project management, this useful guide:

• Explains the relationship of complexity theory to virtual project management
• Supplies techniques, tips, and suggestions for building effective and successful teams in the virtual environment
• Presents current information about best practices and relevant proactive tools
• Makes a strong case for including complexity theory in PMI®'s PMBOK® Guide

Complexity Theory and Project Management gives a firsthand view on the future of complexity theory as a driving force in the management field, and allows project managers to get a head start in applying its principles immediately to produce more favorable outcomes.

(PMI and PMBOK are registered marks of the Project Management Institute, Inc.)

Part I

Complexity Theory

Part I provides an introduction to complexity theory and virtual projects. The first part presents a traditional introduction to the history of complexity history and how complexity theory is used in the business world today. This is followed by a review of major project management associations, their discussions and accommodations of complexity in standards and journals. Virtual projects and leadership are introduced with a short orientation on complexity and how it relates to the distributed environment. Part I ends with examples of successful (the champions) and unsuccessful (the mutts) virtual projects, and how applying complexity increased the likelihood of a champion.

A PRACTITIONER’S EXPLANATION OF COMPLEXITY THEORY

Seasoned project managers realize that all parts of the projects cannot be controlled; nor would they want to have full control of the project. They realize the creativity occurs on the fringes of complexity or chaos. Those teams that appear to be in total chaos may be doing the best work for the project; those work teams that have a catastrophic failure of some sort and are allowed to resolve it on their own do so more quickly and efficiently. This appears to go contrary to standards suggested for project managers.

In the academic sense, complexity theory does go against the common teaching of controlling all aspects of the project. Mounting evidence—both anecdotal and academic—demonstrates that the traditional method just is not working. The Standish Chaos Report (2009) clearly shows that software projects continue to fail more than 65 percent of the time. Yet it is hard to believe the project managers are becoming less competent.

The Standish Report gives the same reasons from report to report. Incompetent project managers are never in the top ten reasons. There is no doubt the reasons are correct but there must be some underlying factor as well. The suggestion is that the seasoned project manager knows how much to control the chaos, what absolutely needs to be controlled, and what can be left to chaos.

This wisdom does not come by chance or happenstance. The project manager must completely understand the theory and implementation of project management. The project manager must understand the integration of the processes and must have the self-confidence of his or her ability as a leader. Once these are in place, then the project manager is ready to embrace the ability to allow chaos on the project.

Chapter 1

Introduction to Complexity Theory

Figure 1.1 The Joshua tree appears to be a hand reaching toward the sky, just as a project manager must reach toward new ideas in order to be successful in the future.

INTRODUCTION TO COMPLEXITY THEORY

Complexity theory evolved from chaos theory, and there has been written evidence of the theory in the scientific community since the 1800s. Relatively speaking, these are very new sciences (Singh & Singh, 2002). Parts of complexity theory are even considered mainstream. Two aspects are commonly referenced in the mainstream media are the butterfly effect and six degrees of separation.

Complexity theory has been alive and well in the areas of math and sciences. Complexity has slowly moved into the areas of social sciences and is now making a step into the world of business (Byrne, 1998). From the social networking perspective (six degrees of separation), complexity theory has made a quantum leap, as discussed in more detail in later chapters. The butterfly effect, if allowed, can be very effective for the virtual project as well. This, too, will be discussed in later chapters.

Complexity theory acknowledges that humans by nature when living or working together are an open system (Byrne, 1998: Hass, 2009). What makes complexity theory different than the traditional open systems theory is that the theory acknowledges that there are parts of the system that cannot be explained but acknowledges that there is normalcy in the randomness. Human beings like to break down the system into its smallest part to explain the whole. Western thought seems content to understand the universe as a series of discreet system rather than a holistic interconnected system. Using this approach to examine, for example, how a single ant works independently, does not explain the dynamics of the colony. Or explaining how the human heart works does not explain the interrelationship of the glands, brain, heart, blood, and so forth, and what happens if one part is out of control. In other words, how will one part of the body compensate for others?

Hence, if complexity is introduced into the system, should it be managed? Think about the ants that are away from the colony on a mission. If an unexpected situation is introduced into the army of ants on the mission or those in the colony, each section of ants begins to react no matter what the situation. Some would say it is instinct. Maybe it is. Without any central authority or predefined processes, the ants resume to their goal. Some of the ants are more central to the goal, while others are more tangential to the goal, but all must perform as a team to ensure the queen’s survival or the heirs’ survival will fail. Is a project any different?

Practical Tip: Try to understand that all systems are connected. By understanding these interconnections, new understanding can be achieved. Explaining matters as discrete silos does not explain the bigger picture. Understanding the role of the quarterback in football does not explain how football is played.

HISTORY OF CHAOS THEORY

For many years, chaos theory was a hobby among mathematicians and scientists. It was not taken seriously because a real world application could not be envisioned. Edward Lorenz, a meteorologist, is credited as the first person to delve into chaos theory. In fact, he is the person credited with the butterfly effect theory. Simply stated, this theory holds that when a butterfly flaps its wings in another part of the world, say South America, this creates a minute disturbance in the atmosphere. This minute disturbance may have a drastic change on the weather conditions in North America. It may create a hurricane or it may prevent a hurricane where one should have started.

What did Lorenz find from all his experiments in the realm of chaos? Many things that do not appear to have any order actually do. Scientists would have called this noise or randomness but it is indeed chaos theory at work. Lorenz’s attractor equation was able to clearly demonstrate order in the world of chaos. He found that the atmosphere never reaches a state of equilibrium. Hence it is always in a state of chaos. When plotting the atmospheric conditions, it always plotted as butterfly wings or owl eyes (Wheatley, 2000). So there was order in the randomness (Figure 1.2). In essence, the atmosphere disturbances were drawn to areas or attractors. Thus, it appeared as order in what previously was thought to be randomness. As a meteorologist, he did not have credibility in the mathematical and physics communities. For many years, his work went unnoticed.

With the introduction of high-speed computers and the observation of chaos in the areas of fluids, semiconductors, and other hard-core science areas, this area of science flourished. Chaos theory remained in the area of hard-core sciences for many years, since it is often difficult to make the leap from one area of academia to another. It would be another ten years or more before enterprising project managers and academics in the area of project management would see the correlation.

Figure 1.2 Lorenz’s Butterfly Attractor Model (Wheatley, 2000)

Chaos theory would still have to go through some changes. It would shift from chaos theory to complexity theory. The theory would shift into various areas of study as it matured. This is no different than what is seen with mature professions such as medicine. Today, there are many different specialties including cardiology, neurology, ophthalmology, and so forth. The human body is probably one of the most complex systems known; even skilled surgeons do not know exactly how each of us will be in surgery. Even if great care is taken prior to surgery, there are no guarantees that the procedure will be successful and without unforeseen complications.

As the area of complexity theory matures, the same would be expected to occur in project management. In fact, the Project Management Institute™ (PMI) has expanded its certification program to include risk and schedule management. Risk is becoming a more widely accepted aspect of both project management and complexity. Although the concept of contingency has existed, this concept supposes that there are controllable unknowns. Complexity accepts that there are simply unknowns and the best manner to handle these would be to have a flexible process rather than a rigid contingency (Weaver, 2007a). This step is the first in accepting that complexity exists in projects, and one can be certain that the future of project management will be more inclusive of this kind of training.

HISTORY OF COMPLEXITY THEORY

Complexity theory grew from chaos theory. The theory works on the notion that a system should not be broken down into fundamental parts to understand the whole. This should not be confused with the theory and studies of self-organizing teams that are seen in the business world.

Complexity theory states that critically interacting components self-organize to form potentially evolving structures exhibiting a hierarchy of emergent system properties. (Lucas, 2006)

Chaos theory offers a view of the universe that everything is not as orderly as once thought. There are many seemingly immutable laws, such as the speed of light, the forward movement of time, and the force of gravity; there seem to be other natural phenomenon that defy this kind of explanation. In some cases, one must understand the entire system in order to understand how all the parts interact. Just as one might understand that the speed of light is 186,000 miles per second, there are still times that light seems to act differently, such as when interacting with a black hole. Time, light, and gravity are seemingly different natural laws, yet there is still a systemic relationship between the three.

As an entity approaches the speed of light, the movement of time apparently slows down, and as light interacts with the intense gravity of a black hole, light seems to move unnaturally. Again, this interaction between three apparently immutable natural laws fundamentally supports the necessity for chaos theory, and by extension, complexity theory.

From another angle, there is simply too much interaction among any natural systems. Weather, which is another chaotic system, is too complicated to fully understand. There are simply too many parts that interact and our understandings of these interactions are based upon observation rather than upon a mathematical model. There are certain truths, such as there cannot be rain without clouds, but there are many instances where this kind of simplistic modeling does not describe the system (Weaver, 2007b). Fundamentally, many natural and human systems are incapable of being described by a mathematical model. Few human systems can be properly described and predicted through single variable mathematical modeling. Too often individuals believe that certain systems are predictable based upon mathematical modeling. This becomes a major stumbling block toward the acceptance of complexity theory. The reality is that with most human-based systems, there are too many variables to predict the results. Just as there is no perfect way to forecast the weather or to predict whether an individual’s favorite soccer team will be victorious in its next game, most human social systems are similarly unpredictable. What often befuddles individuals is that statistical modeling is representative of groups and relationships and not of behavior. Hence, statistical modeling is helpful in understanding groups and relationships but does not help us understand individuals and social systems. This becomes the leadership crux because as a leader, one needs to utilize all the available project management information in order to be successful, but it does not offer a clear path. Simply repeating a known strategy and hoping that it will be successful is not the best course of action.

As complexity theory is maturing, scientists, whether in hard-core science or in business or behavioral aspects, have come to realize that complex systems cannot be viewed broken apart to understand the whole. They have realized the importance of applying complexity theory to business. Think of the old argument of nature versus nurture, or of identical twins separated at birth. Science for many years has not been able to prove what is nature and what is nurture or if it is interrelated. Why do some individuals faced with almost the same familial situations end up with radically different reactions or personalities? While it cannot all be contributed to complexity theory, it would be reasonable to apply complexity theory to the phenomenon.

COMPLEXITY THEORY IN USE TODAY

Several elements of the manifestation of complexity theory are already in use. One of those elements is transformational leadership. In order to understand how transformational leadership differs from leadership of the past, it is necessary to understand the evolution of leadership theory. The history of leadership can be viewed as a continuum of eras leading up to leadership in the virtual environment. Just as society has changed, leadership has changed in a manner to reflect these sociological changes. Some research has tried to categorize leadership into different schools of thought, such as old school and new school; this kind of division creates a feeling that there was a split in thinking. This kind of division is based upon one simple rule of leadership theory of the past and of the present.

In the past, there was an attempt to find the one right way to lead people. This assumption was rooted in management theory for a very long time and whenever a theory came forward that did not cover all the possibilities, it would be rejected and then the next management theory would come forward. Over time more management theories came forward, and often they would reflect society, technology and business. As leadership evolved, more ideas were presented and rejected. The interesting aspect about this is that over time, many of these ideas became integrated into modern thought.

In fact, there is more of an evolution of thought that reflects society, culture, and even business. To this end, leadership has evolved from one stage to the next and in each step of the process certain elements have been carried along to the next. Transformational leadership and its characteristics of mentorship and learnin...

Table of contents

• Title Page
• Copyright Page
• Preface
• Acknowledgments
• Introduction
• Part I - Complexity Theory
• Part II - How to Deploy Complexity Theory
• Part III - Case Studies of Applied Complexity
• Part IV - Create Successful Project Communities
• Part V - ADVANCED TOOLS FOR MANAGING COMPLEXITY
• Index