![]()
Chapter 1
Introduction of Probability Concepts in Physics—the Path to Statistical Mechanics
N. Sukumar
It was an Italian gambler who gave us the first scientific study of probability theory. But Girolamo Cardano, also known as Hieronymus Cardanus or Jerome Cardan (1501–1576), was no ordinary gambler. He was also an accomplished mathematician, a reputed physician, and author. Born in Pavia, Italy, Cardan was the illegitimate son of Fazio Cardano, a Milan lawyer and mathematician, and Chiara Micheria. In addition to his law practice, Fazio lectured on geometry at the University of Pavia and at the Piatti Foundation and was consulted by the likes of Leonardo da Vinci on matters of geometry. Fazio taught his son mathematics and Girolamo started out as his father's legal assistant, but then went on to study medicine at Pavia University, earning his doctorate in medicine in 1525. But on account of his confrontational personality, he had a difficult time finding work after completing his studies. In 1525, he applied to the College of Physicians in Milan, but was not admitted owing to his illegitimate birth. Upon his father's death, Cardan squandered his bequest and turned to gambling, using his understanding of probability to make a living off card games, dice, and chess. Cardan's book on games of chance, Liber de ludo aleae (On Casting the Die, written in the 1560s, but not published until 1663), contains the first ever exploration of the laws of probability, as well as a section on effective cheating methods! In this book, he considered the fundamental scientific principles governing the likelihood of achieving double sixes in the rolling of dice and how to divide the stakes if a game of dice is incomplete.
First of all, note that each die has six faces, each of which is equally likely (assuming that the dice are unloaded). As the six different outcomes of a single die toss are mutually exclusive (only one face can be up at any time), their probabilities have to add up to 1 (a certainty). In other words, the probabilities of mutually exclusive events are additive. Thus, P(A = 6) = 1/6 is the probability of die A coming up a six; likewise P(B = 6) = 1/6 is the probability of die B coming up a six. Then, according to Cardan, the probability of achieving double sixes is the simple product:
The fundamental assumption here is that the act of rolling (or not rolling) die A does not affect the outcome of the roll of die B. In other words, the two dice are independent of each other, and their probabilities are found to compound in a multiplicative manner. Of course, the same conclusion holds for the probability of two fives or two ones or indeed that of die A coming up a one and die B coming up a five. So we can generalize this law to read
provided A and B are independent events. Notice, however, that the probability of obtaining a five and a one when rolling two dice is 1/18, since there are two equally likely ways of achieving this result: A = 1; B = 5 and A = 5; B = 1. Thus
Likewise, the probability of obtaining a head and a tail in a two-coin toss is 1/2 × 1/2 + 1/2 × 1/2 = 1/2, while that of two heads is 1/2 × 1/2 (and the same for two tails) because the two-coin tosses, whether performed simultaneously or sequentially, are independent of each other.
Eventually, Cardan developed a great reputation as a physician, successfully treating popes and archbishops, and was highly sought after by many wealthy patients. He was appointed Professor of Medicine at Pavia University, and was the first to provide a (clinical) description of typhus fever and (what we now know as) imaginary numbers. Cardan's book Arts Magna (The Great Art or The Rules of Algebra) is one of the classics in algebra. Cardan did, however, pass on his gambling addiction to his younger son Aldo; he was also unlucky in his eldest son Giambatista. Giambatista poisoned his wife, whom he suspected of infidelity, and was then executed in 1560. Publishing the horoscope of Jesus and writing a book in praise of Nero (tormentor of Christian martyrs) earned Girolamo Cardan a conviction for heresy in 1570 and a jail term. Forced to give up his professorship, he lived the remainder of his days in Rome off a pension from the Pope.
The foundations of probability theory were thereafter further developed by Blaise Pascal (1623–1662) in correspondence with Pierre de Fermat (1601–1665). Following Cardan, they studied the dice problem and solved the problem of points, considered by Cardan and others, for a two player game, as also the “gambler's ruin”: the problem of finding the probability that when two men are gambling together, one will ruin the other. Blaise Pascal was the third child and only son of Étienne Pascal, a French lawyer, judge, and amateur mathematician. Blaise's mother died when he was three years old. Étienne had unorthodox educational views and decided to homeschool his son, directing that his education should be confined at first to the study of languages, and should not include any mathematics. This aroused the boy's curiosity and, at the age of 12, Blaise started to work on geometry on his own, giving up his playtime to this new study. He soon discovered for himself many properties of figures, and, in particular, the proposition that the sum of the angles of a triangle is equal to two right angles. When Étienne realized his son's dedication to mathematics, he relented and gave him a copy of Euclid's elements.
In 1639, Étienne was appointed tax collector for Upper Normandy and the family went to live in Rouen. To help his father with his work collecting taxes, Blaise invented a mechanical calculating machine, the Pascaline, which could do the work of six accountants, but the Pascaline never became a commercial success. Blaise Pascal also repeated Torricelli's experiments on atmospheric pressure (New Experiments Concerning Vacuums, October 1647), and showed that a vacuum could and did exist above the mercury in a barometer, contradicting Aristotle's and Descartes' contentions that nature abhors vacuum. In August 1648, he observed that the pressure of the atmosphere decreases with height, confirming his theory of the cause of barometric variations by obtaining simultaneous readings at different altitudes on a nearby hill, and thereby deduced the existence of a vacuum above the atmosphere. Pascal also worked on conic sections and derived important theorems in projective geometry. These studies culminated in his Treatise on the Equilibrium of Liquids (1653) and The Generation of Conic Sections (1654 and reworked on 1653–1658). Following his father's death in 1651 and a road accident in 1654 where he himself had a narrow escape, Blaise turned increasingly to religion and mysticism. Pascal's philosophical treatise Pensées contains his statistical cost-benefit argument (known as Pascal's wager) for the rationality of belief in God:
If God does not exist, one will lose nothing by believing in him, while if he does exist, one will lose everything by not believing.
In his later years, he completely renounced his interest in science and mathematics, devoting the rest of his life to God and charitable acts. Pascal died of a brain hemorrhage at the age of 39, after a malignant growth in his stomach spread to the brain.
In the following century, several physicists and mathematicians drew upon the ideas of Pascal and Fermat, in advancing the science of probability and statistics. Christiaan Huygens (1629–1694), mathematician and physicist, wrote a book on probability, Van Rekeningh in Spelen van Geluck (The Value of all Chances in Games of Fortune), outlining the calculation of the expectation in a game of chance. Jakob Bernoulli (1654–1705), professor of mathematics at the University of Basel, originated the term permutation and introduced the terms a priori and a posteriori to distinguish two ways of deriving probabilities. Daniel Bernoulli (1700–1782), mathematician, physicist, and a nephew of Jakob Bernoulli, working in St. Petersburg and at the University of Basel, wrote nine papers on probability, statistics, and demography, but is best remembered for his Exposition of a New Theory on the Measurement of Risk (1737). Thomas Bayes (1702–1761), clergyman and mathematician, wrote only one paper on probability, but one of great significance: An Essay towards Solving a Problem in the Doctrine of Chances published posthumously in 1763. Bayes' theorem is a simple mathematical formula for calculating conditional probabilities. In its simplest form, Bayes' theorem relates the conditional probability (also called the likelihood) of event A given B to its converse, the conditional probability of B given A:
where P(A) and P(B) are the prior or marginal probabilities of A (“prior” in the sense that it does not take into account any information about B) and B, respectively; P(A|B) is the conditional probability of A, given B (also called the posterior probability because it is derived from or depends on the specified value of B); and P(B|A) is the conditional probability of B given A. To derive the theorem, we note that from the product rule, we have
Dividing by P(B), we obtain Bayes' theorem (Eq. 1.2), provided that neither P(B) nor P(A) is zero.
To see the wide-ranging applications of this theorem, let us consider a couple of examples (given by David Dufty). If a patient exhibits fever and chills, a doctor might suspect tuberculosis, but would like to know the conditional probability P(TB|fever & chills) that the patient has tuberculosis given the present symptoms. Some half of all TB sufferers exhibit these symptoms at any point in time. Thus, P(fever & chills|TB) = 0.5. While tuberculosis is now rare in the United States and affects some 0.01% of the population, P(TB) = 0.0001; fever is a common symptom, generated by hundreds of diseases, and affecting 3% of Americans every year, and hence P(fever & chills) = 0.03. Thus the conditional probability of TB given the symptoms of fever and chills is
or about 1.6 in a thousand. Another common situation is when a patient has a blood test done for lupus. If the test result is positive, it can be a concern, but the test is known to give a false positive result in 2% of cases: P(test⊕|no lupus) = 0.02. In patients with lupus, 99% of the time the test result is positive, that is, P(test⊕|lupus) = 0.99. A doctor would like to know the conditional probability P(lupus|test⊕) that the patient has lupus, given the positive test result. Lupus occurs in 0.5% of the US population, so that P(lupus) = 0.005. The probability of a positive result in general is
where we have used the sum rule for mutually exclusive events in the first step, and Equation 1.3 in the next step. The probability of lupus, given the positive test result, is then P(lupus|test⊕) = 0.99 × 0.005/0.02485 = 0.199. So, in spite of the 99% accuracy of the test, there is only a 20% chance that a patient testing positive actually has lupus. This seemingly nonintuitive result is due to the fact that lupus is a very rare...
