1
What is entropy?
1.1 Introduction
Urban and regional models are of interest for two main reasons. Firstly, model building is at the root of all scientific activity, and urban and regional modelling is part of an attempt to achieve a scientific understanding of cities and regions. Secondly, a variety of severe urban and regional problems exist, and associated planning activity has become increasingly important: urban and regional modelling is a part of the advance on this front also.
The urban and regional scientist faces a number of severe theoretical problems. His activity is often a multidisciplinary one in the sense that he needs to use concepts from several disciplines-economics, geography, sociology, and so on. The concept of entropy has, until recently, been used primarily in the nonsocial sciences. This book is an attempt to show that it has a useful and valuable role in one branch of the social sciences—the study of urban and regional systems-and, in passing, the account may stimulate people in other fields also. It is hoped that ‘entropy’ enables the social scientist to tackle some of his basic problems in a fruitful way, and thus to make progress which might not be possible so easily with more orthodox tools.
One of the initial problems is that entropy as a concept can be used in several ways, and so the first task, in section 1.2 below, is to outline three views of the concept. In section 1.3, we shall discuss in general terms how the concept can be used: as a theory building and as an interpretive aid, and to assist in the study of system dynamics. Various examples are then discussed: transport flows (urban and regional) of various kinds in Chapters 2 and 3; location models in Chapter 4; utility maximising systems in Chapter 6. Some technical problems concerned with modelling can be solved using entropy maximising procedures, and these are shown in Chapter 5. There is a general discussion of the relation of the concept to social physics and general systems theory in Chapter 7. A number of conclusions are drawn in Chapter 8.
1.2 Three views of entropy
1.2.1 The relationship of entropy to probability and uncertainty
A ‘system of interest’ in the context of this book is any urban or regional system which it is useful to study. Many examples will be discussed. It is necessary to begin by defining and specifying a state of the system of interest. What information do we need to be given to specify fully a system state? This question can be partly answered by giving one or two examples, though we shall see later that it is a far from trivial question. In physics, and in particular in classical physics, the state of a gaseous system is fully specified by the coordinates and velocities of each particle in the gas at any time. Such a system should be analysed from first principles using the techniques of Newtonian classical physics. However, such techniques (which can be described as microanaly tic for a gas) prove too difficult to handle in many common situations when there are of the order of more than 1023 particles involved. In fact, they are pretty difficult when more than two particles are involved. A branch of physics, known as statistical mechanics, developed in order to study such systems, and new techniques emerged which enabled the physicist to explain and predict certain macroproperties of the system to a desired degree of accuracy without having to explain (at the microlevel) the behaviour of each individual particle. Such macroanalytic techniques are related to microsituations using the concept of entropy (cf. Jaynes, 1957).
Now, consider an equivalent example from the social sciences. Take a system with a fixed spatial distribution of numbers of workers in residences, and a fixed spatial distribution of number of jobs. The system of interest is taken to be the flow of workers from residences to jobs. How would we now specify a state of the system? Simply: it would be necessary to state, for each individual worker, his residential zone and his workplace zone. In a large city, the task of predicting the behaviour of each individual in this respect is too great (as with the corresponding situation in the study of gases). Once again, we may only be interested in certain macroproperties of the system, and it turns out that such properties can be explained and predicted by using statistical techniques which use the concept of entropy.
These examples, then, illustrate the kind of situation where we need to use the concept of entropy, and we now have to face up to the basic question: what is entropy? The second example above, the ‘journey-to-work system’, will be used to illustrate the concept by relating it to the difficult, but perhaps better understood, concepts of probability and uncertainty.
In our example, the system of interest can be specified by an origin-destination table, as shown in Figure 1.1.
Figure 1.1Origin-destination table.
A state of the system is an assignment of individual workers to the origin-destination table which is in accord with any constraints, such as the requirement, mentioned above, that the distribution of numbers of worker residences and jobs is assumed to be given. This is a complete description of the system of interest. We must also define a distribution which is the macroproperty of the system we wish to estimate by statistical means. In this case, a distribution is a set of numbers, one for each origin-destination pair, and one such number is the total number of people who travel from origin i to destination j. Thus each element, or ‘box’, of the origin-destination matrix in Figure 1.1 can be considered to contain two components: firstly, a set of individual workers who travel from i to j (the total collection of such sets being called an assignment of individuals for the whole system, and an assignment defines a single state of the system); and, secondly, a total Tij of individuals travelling from i to j (the set of such numbers being called a distribution of the system).
It can easily be seen that there are many states which give rise to any particular distribution. Suppose we make the assumption that any state of the system occurs with equal probability. Then, we can find the most probable distribution by calculating the set of Tij’s which has the greatest number of states associated with it. This calculation can be carried out without any knowledge being needed of the particular individua...