Some 260 years ago, a little essay was read at a meeting of the Royal Society, not by the author (for the wholly excusable reason that he had died two years previously) but by a friend of his. The title (
An Essay Towards Solving a Problem in the Doctrine of Chances) is unlikely to set the heart of my esteemed readers racing, but the content of the paper was the first step towards one of the major advances in human understanding.
The author, who by birth was as English as they come (born in Tunbridge Wells) could be considered a 'cultural Scot' as he, like many other dissenters, found his intellectual home in the Scottish Enlightenment of the early 1700s, studying logic and theology at the University of Edinburgh (Oxford and Cambridge being barred to anyone unwilling to accept the 39 articles of the Anglican Church). As a Presbyterian minister with a side interest in mathematics and probability, he did not share the anti-theist stance of figures of the Scottish Enlightenment, such as David Hume whose essay
On Miracles comes as close to denying the existence of divine being as was allowed by the laws of the time. (Hume was keen not to follow the fate of the theologian Thomas Woolston who, found guilty in 1729 of blasphemy for denying the existence of miracles, spent the rest of his days in prison.)
It is fair to say that our author, the Reverend Thomas Bayes (1701-1761), was interested in probability theory as an intellectual puzzle and did not understand the deep implications of his own theory. It would take a French mathematician and scientist of the stature of Pierre Simon Laplace to formalise properly and, more importantly, to appreciate the full import of what is commonly known as 'Bayes' theorem' (or 'formula', or 'rule', or 'principle'). Laplace was unencumbered by any religious belief; famously when Napoleon remarked that his book on motion of the solar system contained no references to 'the Creator', he tartly replied that he had no need of such a superfluous hypothesis.
It is one of the mysteries in the history of science why Bayes-Laplace theory of knowledge was ignored for well over a century and only recently has been recognised as a pillar of what has been called 'the Causal Revolution' (to borrow a term from the splendid
The Book of Why by Judea Pearl).
It can be argued that Bayes' theorem provides nothing less than the mathematical validation of the scientific method. In a nutshell, the process of scientific discovery proceeds in four steps: (1) start with a hypothesis you believe to be correct; (2) work out a testable consequence of said hypothesis; (3) perform experiment; (4) update your belief in your hypothesis on the basis of the evidence of the experiment (i.e., revise/reject hypothesis). The tricky part is step (4) because most evidence involves some element of uncertainty.
Bayes' stroke of genius was to look at probabilities in a completely new way. To explain this, we can resort to the example he himself used in his posthumous paper. Suppose you turn your back to a billiard table and throw a ball onto it. You are curious about its eventual resting spot on the table; more specifically, about how far the ball is from the left-hand end of the table. If you know the length of the table (say L feet), the probability that the ball is x feet from the left-hand end is x/L. This is very intuitive and easy for the human brain to compute: the probability of the ball being two feet from the left-hand side is larger if the table is five-feet long (probability 2/5) than if the table is 10-feet long (probability 2/10).
But Bayes was interested in a different problem (what at the time and for a long period was called 'the inverse probability'). Suppose you are told that the ball is resting three feet from the left-hand side, but you are not told how long the table is. Can you work out the probability that the table is L-feet long?
My guess is that you will find this problem a lot harder than working out the probability of the ball being x feet from the left-hand side when you knew the length of the table: computing the probability of x given L is easier than computing the probability of L given x. Why the asymmetry? Because the human brain works from cause to effect, not from effect to cause. If we see a deranged criminal pointing a loaded gun towards someone (the cause) we can easily predict that the effect will be someone being shot. But if we see someone with a gunshot wound (the effect), working out the cause is a lot more complicated.
Unknown to him, the Revered Thomas Bayes managed to resolve this cognitive asymmetry and in the process gave us the most efficient method for improving our knowledge.
If this discussion seems a bit abstract, let me apply Bayes' principle to a matter of life or death. Suppose you suspect you may have bowel cancer and your doctor suggests you take a diagnostic test which will correctly identify the disease with a 99% accuracy, whereas if you do not have the disease it will give a false positive with a 10% probability. Effectively, the doctor is telling you that if you have bowel cancer (the cause), the effect will be a positive test (with 99% probability). From cause to effect.
But this is not what you are interested in. What you really want to know is this: if the test comes out positive (the effect), how likely am I to have the disease (the cause)? From effect to cause.
Note two important features of this story: the 99% accuracy of the test is an objective fact, based on the technology of the testing equipment and is the same for all types of patients. The really key issue is how relevant a positive test is for you to assess the probability that you have bowel cancer. More precisely, before taking the test, your prior assessment of you having cancer would have been based on the latest statistics for people of your gender and age, giving you a 3% probability of having bowel cancer. After taking into account other factors, e.g., your genetic history and your symptoms, you conclude that your subjective prior probability of having the disease is, say, 6%.
But after a positive test, you have a new piece of information. Surely, now your updated probability of having the disease must have increased. But by how much, precisely?
Faced with a positive result from a nearly infallible test, most people (and according to multiple experiments this includes doctors treating bowel cancer) would conclude that almost certainly your probability of having bowel cancer must very close to one.
Bayes (as formalised by Laplace) gives us the perfect and beautifully simple one-line formula for revising our prior belief in the light of the new data: the new updated probability of having the disease is equal to our old prior probability multiplied by a simple ratio (the 'likelihood ratio'). This ratio tells us how much more likely patients with the disease are to test positive compared with the general population.
Using the data in the example, it turns out that a positive result from a nearly infallible test does increase the probability of having the disease (it would be incredible if it did not) but not to near certainty, rather to a more reassuring probability of less than 48%.
The reason for the discrepancy between the intuitive answer (near certainty of having bowel cancer) and the correct answer (more likely not to have bowel cancer) is that most people disregard the fact that for relatively rare diseases many more healthy patients will have a false positive test than the few ill patients testing positive, thereby reducing substantially the value of the test results in updating one's prior beliefs.
What general lessons can we learn from Bayesian decision making?
First, do not trust your intuition. Second, keep an open mind: when you do not have a lot of information (and before the arrival of new data you do not, by definition) do not come to firm conclusions. Keep your prior probabilities away from extremes (either zero or one). Third: when updating your beliefs, make sure to consider all the relevant evidence, not just what supports your prior. Fourth: do not waste your time trying to change the opinions of people whose minds are closed. If your friend is certain that alien abductions are commonplace, no amount of evidence to the contrary will ever shift his or her prior. Fifth: if your prior is based on solid and verified data and has increased through time on the basis of new evidence (i.e., it is Bayesian-good), any evidence that questions it must be extraordinarily strong.
Dr Manfredi La Manna is a Reader in Economics at the University of St Andrews
