Class 4.2: Bayes Theorem

Methodology of Scientific Research

Andrés Aravena, PhD

7 April 2021

Bayes Theorem

“An Essay towards solving a Problem in the Doctrine of Chances” is a work on the mathematical theory of probability by the Reverend Thomas Bayes, published in 1763, two years after its author’s death

The use of the Bayes theorem has been extended in science and in other fields

From Wikipedia, the free encyclopedia

Bayes rule

Since \[ℙ(A, B) = ℙ(A)⋅ ℙ(B|A)\] and, by symmetry \[ℙ(A, B) = ℙ(B)⋅ℙ(A|B)\] then we can reorganize everything as \[ℙ(B|A) = \frac{ℙ(B)⋅ℙ(A|B)}{ℙ(A)}\]

One of the most famous theorems

What does it mean

It can be understood as \[ℙ(B|A) = ℙ(A|B)⋅\frac{ℙ(B)}{ℙ(A)}\] which is a rule to invert the conditional probability

This is the view we will use now

How do we learn

Another point of view is \[ℙ(B|A) = \frac{ℙ(A|B)}{ℙ(A)}⋅ℙ(B)\] which is a rule to update our opinions

  • \(ℙ(B)\) is the a priori probability
  • \(ℙ(B|A)\) is a posteriori probability

Bayes says how to change \(ℙ(B)\) when we learn \(A\)

“When the facts change, I change my opinion. What do you do, sir?”

John Maynard Keynes (1883 – 1946), English economist, “father” of macroeconomics

We need ℙ(A)

We can write \(ℙ(A)\) as \[ℙ(A) = ℙ(A,B) + ℙ(A,\textrm{not }B)\] which can be rewritten as \[ℙ(A) = ℙ(B)⋅ℙ(A|B) + ℙ(\textrm{not }B)⋅ℙ(A|\textrm{not }B)\] therefore \[ℙ(B|A) = \frac{ℙ(B)⋅ℙ(A|B)}{ℙ(B)⋅ℙ(A|B) + ℙ(\textrm{not }B)⋅ℙ(A|\textrm{not }B)}\]

Diagnostics Using Bayes Theorem

We want to evaluate the probability of having COVID, given that the test is positive \[ℙ(\text{COVID}_+ | \text{test}_+)\] given that we know the prevalence \(ℙ(\text{COVID}_+)=0.001\) and the precision \[ \begin{aligned} ℙ(\text{test}_+ | \text{COVID}_+)=0.99\\ ℙ(\text{test}_- | \text{COVID}_-)=0.99 \end{aligned} \]

What is the probability of being sick given that the test is positive?

\[ℙ(\text{COVID}_+ | \text{test}_+)=\frac{ℙ(\text{test}_+ | \text{COVID}_+)⋅ℙ(\text{COVID}_+)}{ℙ(\text{test}_+)}\]

We need to know \(ℙ(\text{test}_+),\) which we can get as

\[ℙ(\text{test}_+)= ℙ(\text{test}_+, \text{COVID}_+)+ ℙ(\text{test}_+, \text{COVID}_-)\]

We can calculate them

\[\begin{aligned} ℙ(\text{test}_+, \text{COVID}_+)& =ℙ(\text{test}_+ \vert \text{COVID}_+)\cdotℙ(\text{COVID}_+)\\ & =0.99⋅ 0.001 \end{aligned} \]


\[ \begin{aligned} ℙ(\text{test}_+, \text{COVID}_-)& =ℙ(\text{test}_+ \vert \text{COVID}_-)\cdotℙ(\text{COVID}_-) \\ & =(1-0.99)⋅ (1-0.001) \\ & =0.01⋅ 0.999 \end{aligned} \]

The last part is true, because

\[ \begin{aligned} ℙ(\text{test}_+ \vert \text{COVID}_-) & =1-ℙ(\text{test}_- \vert \text{COVID}_-)\\ & =(1-0.99) =0.01 \end{aligned} \] and \[ \begin{aligned} ℙ(\text{COVID}_-) & =1-ℙ(\text{COVID}_+)\\ & =(1-0.99)⋅ (1-0.001) \\ & =0.01⋅ 0.999 \end{aligned}\]

Therefore \(ℙ(\text{test}_+)= 9.9\times 10^{-4} + 0.00999=0.01098\)

Finally, we find the value we were looking for

\[ \begin{aligned} ℙ(\text{COVID}_+ \vert \text{test}_+) & = \frac{ℙ(\text{test}_+, \text{COVID}_+)}{ℙ(\text{test}_+)}\\ & = \frac{0.99\cdot 0.001}{0.01098} \\ & = 9.02\% \end{aligned} \]


We would like to have no errors, but errors are usually unavoidable. The technology used will limit the precision of diagnostics.

In practice, diagnostic test designers know that telling that you are healthy when you are sick is much worse than telling that you are sick when you are healthy. In other words, the cost of a false positive is usually lower than the cost of a false negative.

Of course the costs depend on the disease and in the technique.

What happens if the false positive rate is different from the false negative rate?

\[\begin{aligned} ℙ(\text{test}_+ \vert \text{COVID}_+)=a\\ ℙ(\text{test}_- \vert \text{COVID}_-)=b \end{aligned} \]