Stochastic Dynamics of Complex Systems

From Glasses to Evolution

Paolo Sibani, Henrik Jeldtoft Jensen

• Publisher: ICP
• Year: 2013
• ISBN:
  9781848169951
• Topic: Physical Sciences
• Subtopic: Chaotic Behavior in Systems

Dynamical evolution over long time scales is a prominent feature of all the systems we intuitively think of as complex — for example, ecosystems, the brain or the economy. In physics, the term ageing is used for this type of slow change, occurring over time scales much longer than the patience, or indeed the lifetime, of the observer. The main focus of this book is on the stochastic processes which cause ageing, and the surprising fact that the ageing dynamics of systems which are very different at the microscopic level can be treated in similar ways.

The first part of this book provides the necessary mathematical and computational tools and the second part describes the intuition needed to deal with these systems. Some of the first few chapters have been covered in several other books, but the emphasis and selection of the topics reflect both the authors' interests and the overall theme of the book. The second part contains an introduction to the scientific literature and deals in some detail with the description of complex phenomena of a physical and biological nature, for example, disordered magnetic materials, superconductors and glasses, models of co-evolution in ecosystems and even of ant behaviour. These heterogeneous topics are all dealt with in detail using similar analytical techniques.

This book emphasizes the unity of complex dynamics and provides the tools needed to treat a large number of complex systems of current interest. The ideas and the approach to complex dynamics it presents have not appeared in book form until now.

Contents:

• Complex Dynamics: Tools and Applications:
  • Characterization of Collective Dynamics
  • Markovian Stochastic Processes
  • Monte Carlo Methods
  • Record Statistics and Extremal Statistics
  • Complexity and Hierarchies
  • Energy Landscapes
  • Record Dynamics and Marginal Stability
• Complex Systems with Similar Dynamics:
  • Ageing of Spin Glasses
  • Magnetic Relaxation in Superconductors
  • Ageing of Colloids
  • Evolving Biological Systems
  • Non-stationary Ageing Dynamics in Ant Societies
• Epilogue:
  • What is Complexity Science?

Readership: Graduates and researchers with an interest in complex systems.

1

Introduction

The general view behind this book is that complexity science is a meaningful field of endeavour in which stochastic processes play an essential and unifying role. We therefore develop from scratch the methods and techniques needed to model a body of experimental and numerical data behind complex phenomena. These methods and techniques are then applied to a range of physical and biological problems of current interest.

HIERARCHIES, COMPLEXITY AND DYNAMICS

That science has an overall hierarchical structure, often related to characteristic length, time and energy scales, is fairly uncontro-versial. Within this structure, many natural phenomena are only associated to a single or a few scales and can hence be studied in relative isolation. This fact underlies the success of the reductionist approach common in physics, where branches belonging to different levels of description have largely been developed independently, e.g. the elastic properties of an iron bar can be described without knowledge of nuclear physics, because nuclear processes and sound propagation occur on widely separated energy and length scales. While the particular organization of elementary particles making up the bar is the result of a series of dynamical processes, starting from nuclear reactions in a star and ending with its forging from an iron melt, all these processes are of little relevance to elasticity.

In contrast, energy and length scales are less broadly distributed and more closely intertwined in biology, e.g. a multicellular organism has itself a hierarchical structure: The dynamics of macromolecules leads to the formation of organelles and other cellular building blocks. The dynamical interactions of cells produces organs and all the different types of tissue. The entire structure of tissue and organs—helped by external agents such as bacteria—brings about a biological organism, e.g. a mouse or a man. Dynamics on much longer time scales is also important for structure: In the words of Theodosius Dobzhansky [Dob73], “Nothing in biology makes sense except in the light of evolution.”

Darwinian evolution itself is a prime example of complex dynamics. First of all, it is inherently stochastic, i.e. it is driven by random mutations. Successful mutations owe their success to interaction patterns between individuals and between individuals and their environment. These patterns are mainly the outcome of historical contingencies and hence themselves random, to a degree. Secondly, even though in its original formulation Darwinian evolution only deals with individual organisms, the distinction between an individual and a tightly knitted ecosystem appears now blurred, and evolutionary principles are often applied at the level of species or even higher taxa. Hence, evolutionary dynamics is a random process which unfolds in a hierarchically organized structure, starting with cells at the lowest level and ending with, e.g., genera [Gou02].

While an engineer interested in the stiffness of a bicycle frame can safely neglect that the frame can melt, the situation differs in biology (and sociology and economics and neuroscience etc.), where phenomena at different scales do interact more strongly. Very small vira (size 10^–8 – 10^–6 m) can affect much larger living organisms, e.g. a blue whale, (linear size 10 m) or an Aspen cluster, (linear size 10³ m). This possibility arises because, irrespective of body size, all cells have similar biochemical and biophysical properties. Furthermore, organisms are more than the sum of their parts: The functioning of the brain is not immediately reducible to that of its neurons. In vitro analyses may therefore miss important aspects of the in vivo behaviour, where several levels of a description hierarchy are inextricably mixed.

COMPLEX DYNAMICS

While a reductionist approach strives toward understanding and controlling what lies below the phenomena observed at some level in a hierarchical description of nature, the syncretic approach more common in complexity science strives to identify collective or emergent properties. Both procedures lead to new ontological levels: Since the times of Leucippus and Democritus the meaning of indivisible (atom) has been revised downwards many times and new particles have been identified along the way. Conversely, the fundamental frequency of a violin string is not visible at the level of nuclei, nor is the temperature defined at the level of a single degree of freedom. Both quantities are emerging properties at a description level involving a very large number of interacting degrees of freedom.

To move either down or up in the hierarchy of natural phenomena requires different methods. Moving down entails isolating the phenomenon to be described, by fixing a large number of ‘external’ parameters, e.g. optical tweezers can be used to study the elastic properties of a DNA molecule tethered to a bead. In contrast, the description at the emergent level involves the identification of relevant coarse-grained variables, whereby huge amounts of information are neglected, e.g. to describe the elastic vibrations of a violin string, the field of elastic deformations is needed but not the positions of the individual atoms.

Coarse-graining permeates much of physics, including areas not conventionally associated to complexity, e.g. ice and vapour contain the same water molecules but have widely different mechanical properties. Hence, the differences do not stem from the individual components, but from the nature of their interactions. Describing the phase diagram of water is a task for equilibrium statistical mechanics. The latter relies on postulating that microscopic configurations with the same energy are equiprobable, a probabilistic principle which coarse-grains away individual molecular trajectories. Finding emerging properties does in general require statistical methods and probability theory, elements which both embody some kind of coarse-graining in a physical modelling context.

Complex dynamics as defined in this book arises when processes belonging to many different levels of a hierarchy are intermingled and all contribute, in the long run, to a class of macroscopic phenomena. We argue that complex dynamical phenomena have important similarities straddling physics and biology.

That dynamics is an essential part of complex phenomena is the guiding principle of Self-Organized Criticality (SOC) [BTW87, Jen98, Pru11], a paradigm inspired by equilibrium critical phenomena, and aspiring to an all-encompassing generality. SOC emphasises that large classes of driven systems generically enter a stationary state whose real space structure is scale invariant. In contrast to critical phenomena, this happens with no need to tune any external parameters, whence the ‘self-organized’ part of the name. SOC focusses on the properties of the broad size distribution of certain intermittent events, called avalanches. These spatial rearrangements bring the system from one of its critical states to another. Similarly to SOC, we propose that a specific class of rare and intermittent events, which we call quakes, control the dynamics of many complex systems. Unlike SOC, the systems of interest to us are in a non-stationary and ever evolving state. We are interested in a statistical characterization of both the temporal distribution of the quakes and of the system changes they trigger, and claim that these properties are largely insensitive to the particulars of the system at hand. We are not overly concerned with self-similarity under real space dilation, although self-similarity turns out to be of importance. In general, the interactions between the parts of a complex system generate dynamical processes characterized by a hierarchy of time scales [Sim62]. In some cases the latter can be linked to a hierarchy in real space, in others to a hierarchy in a high dimensional configuration space, and in others again to an organizational hierarchy.

A hierarchy of length scales is of importance in critical phenomena: At the critical temperature, i.e. the temperature above which the distinction between liquid and gas is no longer meaningful, density fluctuations are present at all length scales. We can think of a large droplet of liquid water containing vapour, containing smaller droplets of liquid and so on, ad infinitum, or, more precisely, down to the molecular length scale.

A hierarchy of energy and time scales characterizes ‘glassy’ materials. As in SOC, the situation arises for a range of control parameters, and does not presuppose the fine-tuning of, e.g., the temperature. Glassy materials never reach true thermal equilibrium, but are instead in a permanent state of slow flux, a process known in the literature as physical ageing. Ageing is associated to a number of interesting properties, e.g. intermittency and so-called memory and rejuvenation effects. Intermittent events, the quakes just mentioned, accompany qualitative changes in the measurable properties of an ageing system. Successive quakes are separated by gradually longer periods of time during which the system appears quiescent at the level of macroscopic variables.

Biological evolution can be studied at the largest scales, millions of years, by analysing palaeontological data. On much shorter scales, years and even months, one can study bacterial and viral evolution. Finally, one can construct theoretical models and study them on the computer. No matter what the scale is, it is fairly clear that biological structures are metastable, i.e. evolve in time. At the level of say, individuals, evolution is a fairly continuous process involving small changes only. However, at the macroscopic level, say species, changes are rapid (on geological scales) and intermittent, and may involve large-scale reorganization. Some events, e.g. mass extinctions, are clearly due to causes external to the system’s own dynamics, such as meteorite impacts. It is then of interest to study the reaction to the randomising effect of events which typically disrupt, to a lesser or greater extent, the pre-existing organization of an ecosystem. The tempo of evolutionary processes might be decreasing on palaeontological scales, and at least for computer models, there is a striking statistical similarity between the quakes leading a physical system from one metastable configuration to another and the analogous punctuations in an evolving biological system.

EMERGENT PROPERTIES IN COMPLEX SYSTEMS

One way to investigate the phenomena emerging at system level is to develop all-encompassing models and simulate them on a computer. Starting from the level of the individual component one attempts to include as many details as possible. Refined techniques are then used to compute the temporal evolution of huge numbers of degrees of freedom. The aim is to establish a faithful representation of the system considered, say transport through a major city, which allows virtual experiments to be carried out, leading to a detailed phenomenology. In our specific example one might try to collect minute information about the behaviour and needs of the citizens, e.g. weather conditions, fluctuations in fuel prices and their influence on transport preferences, etc. These simulations map out what to expect under given circumstances, which is obviously, a strength. The weakness is that the emphasis on high precision makes it difficult to discriminate between essential and less essential modelling features. This may in turn obfuscate which findings are of general relevance and which are only relevant for the specific situation at hand.

Simulation strategies designed to identify the most important mechanism responsible for certain phenomena and thereby hopefully point to more general aspects often start from bold and simplistic assumptions. In physics, this approach has been very successful for manifold reasons. Firstly, as earlier discussed, one may only need to focus on a few effective degrees of freedom, e.g. when studying the acoustics of a concert hall, one needs not worry about the electronic state of the air molecules. In fact one doesn’t even need to worry about the molecular nature of the air, but can make use of a continuum description, e.g. fluid dynamics. This leads to a representation in terms of densities and waves rather than a representation in terms of individual air molecules. As a bonus, the study of reflection and refraction of waves in the concert hall may also be relevant to other forms of wave propagation.

While both types of approach are needed, they will typically serve different purposes. Detailed simulations can have great practical value, e.g. weather prediction. The more simplistic representation emphasizes the essential aspects allegedly responsible for the observed behaviour, and may elucidate aspects of relevance to a larger class of problems.

The anticipation that similarities exist across complex systems which greatly differ at the microscopic level is supported by our experience from mathematics and physics. An example of a greatly general principle, or law, that is equally relevant to, say, physics, sociology and biology is the Central Limit Theorem: When suitably scaled, the sum of many independent random variables has a normal probability distribution. This kind of universal behaviour comes about because the systemic variables, say the body mass of a biological organism, sum up the contributions from all the parts comprising the total. The macroscopic or systemic behaviour is a consequence of mathematics and will not depend on specific details. Clearly such ‘laws’ are only applicable if the relationship between the components and the collective macroscopic level for a given system is in fact consistent with the assumptions underlying the mathematical result. In the case of the Central Limit Theorem applicability demands sufficiently weak interactions between the constituents.

Phase transitions display a spectacular example of similarity between very different physical systems. The behaviour can be understood in terms of the Renormalization Group (RG) analysis, a quantitative way to establish what the relevant system components are at different length scales, and how these components interact with one another. As earlier mentioned, at a specific value of certain parameters, e.g. temperature and pressure, the macroscopic behaviour at large length scales turns out to be the same irrespective of microscopic details. These lessons from mathematics and physics suggest that at the collective systemic level complex systems of different origin may well look similar. In our view, complexity science has a role to play in the identification and study of such emergent regularities in systems far from equilibrium.

We expect the study of emergent phenomena and the identification of generalities across different classes of problems to be more challenging in biology than in physics. Biological systems are always evolving and their description is intrinsically a stochastic process, as is the case in Darwinian evolution. We also expect that any ‘law’s for complex system dynamics will be statistical in nature, and thus be similar in character to Ohm’s law, which is a relation between macroscopic quantities, current and voltage, and which comes about as a result of averaging over the scattering of the individual electrons as they collide with inhomogeneities in a conducting material.

THE REST OF THIS BOOK

The first part of the book, which includes Chapters 1 to 8, is mainly dedicated to the development of general concepts and methods within statistical data analysis and stochastic processes. These chapters contain exercises, examples and problems. The first three can be used for a short introductory course on stochastic processes in the natural sciences. The following five introduce more advanced topics, which are either applications or developments of the material already presented. Taken together, the eight chapters are suitable for a one semester course.

In the second part of the book, each of Chapters 9 to 13 contains a brief monographic description of a different area in complex science. The topics are selected because of the author's personal involvement and interest, and because taken together, they buttress the claim implied by the book’s title on the unity of complex dynamics. The material in the second part is more advanced. It can serve as an introduction to the scientific literature and it can be used for graduate level courses in complex system dynamics.

All chapters start with a brief introductory section. These sections provide a synopsis of the book’s contents which does not rely on mathematics and which can be read independently of the rest.

Finally, a technical note: Spelling is mainly British, but the original American spelling is kept in quotations, in the bibliography and in figure texts. Numerical simulation results are usually expressed in terms of dimensionless quantities. In other cases, units are specified as needed.

COPYRIGHTS AND CREDITS

The forty figures included in the book are meant to illustrate central points of the material. Some were especially prepared, but others are either reprinted or adapted from previously published material. We extend our grateful thanks to the copyright holders, authors and publishers, for kindly and promptly granting us the permission to reproduce their figures. A full bibliographic reference to the original publication is always given in the caption of a reproduced figure.

Figures reproduced from Physical Review Letters, Physical Review B and Physical Review E are all copyrighted by the American Physical Society in the year of their publication. Readers may view, browse, and/or download these figures for temporary copying purposes only, provided these uses are for noncommercial personal purposes. Except as provided by law, the figures may not be further reproduced, distributed, transmitted, modified, adapted, performed, displayed, published, or sold in whole or part, without prior written permission from the American Physical Society. Figures reproduced from EPL are copyrighted by EDP Sciences in the year of their publication. Figures reproduced from the Journal of Statistical Physics and the Journal of Physics: Condensed Matter, are copyrighted by IOP Publishing Limited, Bristol, UK, in the year of their publication. For figures reproduced from a publication co-authored by Paolo Sibani in the Journal of Theoretical Biology, the copyright holder Elsevier grants the “right to … re-use” them “in other works, with full acknowledgement of its original publication in the journal“. See http://www.elsevier.com/wps/find/authorsview.authors/rights.

Part I

Complex Dynamics:...