This chapter introduces Bayesâ Theorem, named in honour of the Reverend Thomas Bayes. Bayesâ Theorem offers a way to update the probability of a hypothesis being true, given some new evidence, using a simple but very powerful mathematical equation. Bayesian updating is in this way a solution to the problem of how to combine pre-existing (prior) beliefs with new evidence. We also introduce the Bayes Factor, which is the ratio of the likelihood of one hypothesis to the likelihood of another. It is essentially a measure of which hypothesis better explains the world, given the evidence. We examine the Prosecutorâs Fallacy and Laplaceâs Rule of Succession and show some applications of Bayesian reasoning. These include the classic taxi problem, the beetle problem and the false positives problem, the latter taking us into the realms of health and medicine. We also look at the application of Bayesian reasoning in the real-world courtroom. Stylised examples include the Bayesian detective, the Bobby Smith problem, and Bayes at the theatre.
1.1 Bayesâ Theorem: The Most Powerful Equation in the World
How should we change our beliefs about the world when we encounter new data or information? A theorem bearing the name of Thomas Bayes, an eighteenth-century clergyman, is central to the way we should answer this question.
The original presentation of the Reverend Thomas Bayesâ work, âAn Essay toward Solving a Problem in the Doctrine of Chancesâ, was given in 1763, after Bayesâ death, to the Royal Society, by Bayesâ friend and confidant, Richard Price.
In explaining Bayesâ work, Price proposed, as a thought experiment, the example of a person who enters the world and sees the sun rise for the first time. Perhaps he has spent his entire life entombed in a dark cave. As this person has had no previous opportunity to observe dawn, he is not able to decide whether this is a typical or unusual occurrence. It might even be a unique event. Every day that he sees the same thing happen, the degree of confidence he assigns to this being a permanent aspect of nature increases. His estimate of the probability that the sun will rise again tomorrow as it did yesterday and the day before, and so on, gradually approaches but never quite reaches 100%.
The Bayesian viewpoint is just like that, the idea that we learn about the world and everything in it through a process of gradually updating our beliefs. In this way, we edge closer to the truth as we obtain more data, more information, more evidence.
The Bayes Business School, formerly City University of Londonâs business school, explained their choice of name in similar terms: âBayesâ theorem suggests that we get closer to the truth by constantly updating our beliefs in proportion to the weight of new evidence. It is this idea ⌠that is the motivation behind adopting this nameâ (Significance, June 2021, p. 3).
As such, the perspective of Reverend Bayes differs from that of philosopher David Hume. For Hume, assumptions about the future, such as that the sun will rise again, cannot be rationally justified based simply on the past because no law exists that the future will always resemble the past. Bayes instead sees reason as a practical matter, to which we can apply the laws of probability in a systematic way.
To Bayes, therefore, we step ever nearer to the truth based on new evidence and the proper application of the laws of probability. This is called Bayesian reasoning. According to this approach, we can see probability as a bridge between ignorance and knowledge. Bayesâ Theorem is, in this way, concerned with conditional probability. It tells us the probability, or updates the probability, that a theory or hypothesis is correct, given that we observe some new evidence. A particularly good thing about Bayesian reasoning is that the mathematics of it is so straightforward.
At its heart, then, Bayesâ Theorem allows us to use all the information available to us. Our beliefs, our judgments, our subjective opinions, what we have already learned from the previous body of knowledge to which we have had access. We can incorporate this in updating our estimate of the probability that a hypothesis is true. As such, we can be explicit and open about the uncertainty in our data and our beliefs. The problem with implicit reasoning, or intuition, is that our intuition is often wrong and subject to systematic biases. Instead, we should be trained to think in a Bayesian way about the world.
Often the conclusions generated by the application of Bayesâ Theorem will challenge intuition. This is because the world is, in many ways, a counter-intuitive place. Accepting that fact is the first step towards mastering lifeâs logical maze.
Intuition also often lets us down because our in-built judgment of the weight that we should attach to new evidence tends to be skewed relative to pre-existing evidence.
New evidence also tends to colour our perception of the pre-existing evidence. Moreover, we tend to see evidence that is consistent with something being true as evidence that it is in fact true. Bayesâ Theorem is the map that helps guide us through this maze.
Essentially, though, Bayesâ Theorem is just an algebraic expression with three known variables and one unknown. Yet this simple formula is the foundation stone of that bridge between ignorance and knowledge, which can lead to critical predictive insights. Bayesian reasoning allows us to use this formula to update the probability that a theory or hypothesis is true when some new evidence comes to light.
There are three things a Bayesian needs to estimate.
- A Bayesianâs first task is to assign a starting point probability to a hypothesis being true before some new evidence arises. This is known as the âpriorâ probability. Letâs assign the letter âaâ to this.
- A Bayesianâs second task is to estimate the probability that the new evidence would have arisen if the hypothesis was correct. This is sometimes known as the âlikelihoodâ. Letâs assign the letter âbâ to this.
- A Bayesianâs third task is to estimate the prob...
