Routes to Interdisciplinarity 4: The Entropy Story

Sir AlanWilson FBA, FRS, is Director of Research at The London Interdisciplinary School. These blog posts are developed from his books - Knowledge power and Being interdisciplinary. The latter is free to download: www.uclpress.co.uk/beinginterdisciplinary.

‍

Discovery of Entropy in Transport Modelling

I discovered ‘entropy’, and its beginnings as a superconcept, through a combination of an accident of circumstances and through some apparently disconnected piece of knowledge from my student days. After many attempts, I managed to find a job in the social sciences – in fact, in economics, that allowed me to switch from theoretical physics.

I was in the Institute for Economics and Statistics inOxford to work on a research project on cost-benefit analysis (CBA) in relation to investment in transport infrastructure. This needed an account of transport flows between origins and destinations and my task was to build a computer model to achieve this.  

The existing models had been mainly developed in the United States and were based on an analogy with Newton’s law of gravity. That was my starting point. Unfortunately, the basic model was inconsistent in that when the flows from an origin were summed, they did not add up to the known total number of flows from an origin; and similarly for flows into destinations.

The engineers solved this problem by adding ‘fudge factors’ to the flow equations. This was the basis of a lucky break for me. As a student, I had spent serious time on statistical mechanics and in particular on Boltzmann’s theory of gases and I recognised these factors as something like – not exactly like – the partition functions used by Boltzmann.

I recognised that there was a possibility of shifting the underpinnings of the flow model from its Newtonian basis to Boltzmann. I reworked the mathematics of classical statistical mechanics to produce a consistent flows’ model and this was achieved by maximising an entropy function, generating the fudge factors from a formal procedure.

This became a standard method for producing - renamed from the flows’ model – a spatial interaction model.

‍

Boltzmann's Legacy: From Gases to Urban Dynamics

It is worth trying to describe the key idea without the maths. Boltzmann was dealing with systems of gases made up of very large numbers of atoms or molecules, far too large a number to model the individual states of these elements. What he was able to do, was to characterise more aggregate properties of the system such as temperature and pressure and show how they could be calculated from a whole set of individual states.

He then considered an ensemble of system states and showed that of an astronomical number of these, in terms of the aggregate properties, one state was overwhelmingly the most probable. In the transport case, each individual can be assigned an origin and a destination, and the ensemble is the collection of these for all individuals making trips.

An extreme example, would be the case where all individuals were assigned to the same origin-destination pair, clearly highly improbable. However, by finding the most probable system state, and recognising that this is overwhelmingly the most probable, the array of origin-destination flows can be calculated.

This is done through an optimisation process, and it turns out that, in physics, the quantity to be optimised is the entropy; and hence this is the name that is still used in building a transport model. If W is the number of system states that can be generated from a particular assignment of origin destination pairs, then S = klogW is the equation which is the formal representation of the quantity to be maximised. S is a measure of entropy in thermodynamics.  

This equation is engraved on Boltzmann’s grave and is precisely the basis of transport modelling, with different definitions of the ‘elements’ – people instead of particles.

‍

The Universality of Entropy: A Superconcept Beyond Disciplines

The idea, in the first instance, was developed as an analogy, and might seem that transport modelling is based on physics. However, it turns out that the entropy-maximising process is general in the sense that it can be applied to any system with a large number of elements that are only weakly connected – particles in a gas, people in a city, and many more.

It is a superconcept that sits above disciplines. Relatively quickly, I realised that the method could be applied more widely in urban analytics, and later, in areas as diverse as ecology, archeology and history. There is another lesson in developing research ideas here: if you happen to stumble on a good idea, work out how it can be applied much more widely than you might have originally thought.

‍

Bibliographic note.

The original idea was published in Wilson (1967) A statistical theory of spatial distribution models,Transportation Research, 1, pp. 253-69; and in Wilson (1970) Entropy in urban and regional modelling, Pion, London (re-published by Routledge in 2011). Peter Gould wrote a ‘pedagogic review’ of the book in the Annals of the Association of American Geographers, 62, pp. 689-700, which sets out to explain the idea to a wider, less mathematical, community.

The history of its application in urban analytics is recorded in detail in D. Boyce and H. Williams (2015) Forecasting urban travel, Edward Elgar, London, note Chapter 3, pp 90-99. There was a special issue of Geographical Analysis (2010) recognising the 40^th anniversary of the publication of‘Entropy in urban and regional modelling’, which included Wilson (2010) Entropy in urban and regional modelling: retrospect and prospect. Geographical Analysis, 42, pp. 364-394; ; a more recent personal review in a book recognising the 50^th anniversary,  is Wilson (2021)Entropy in urban and regional modelling, Handbook on Entropy, Complexity and Spatial Dynamics: A rebirth ofTheory, (Editors: Aura Reggiani, Laurie A. Schintler and Sammy Czamanski), pp. 20-31.