Bayes’ Theorem is a fundamental concept in probability theory, named after the Reverend Thomas Bayes, an 18th-century British mathematician and theologian. It provides a way to calculate the probability of an event, given some prior knowledge or evidence about that event.
The theorem is based on the idea of conditional probability, which is the probability of an event occurring given that another event has occurred. Bayes’ Theorem provides a way to calculate this conditional probability, which is useful in many fields, including statistics, data science, and machine learning.
Bayes Theorem Formula
P(A|B) = P(B|A) * P(A) / P(B)
where:
- P(A|B) is the conditional probability of event A given event B
- P(B|A) is the conditional probability of event B given event A
- P(A) is the prior probability of event A
- P(B) is the prior probability of event B
In other words, Bayes’ Theorem states that the probability of A given B is equal to the probability of B given A times the prior probability of A, divided by the prior probability of B.
How Bayes’ Theorem?
To understand how Bayes’ Theorem works in practice, consider an example. Suppose that a hospital has a test to diagnose a certain disease, and the test has a 95% accuracy rate. This means that if a person has the disease, the test will correctly identify them as having the disease 95% of the time, and if a person does not have the disease, the test will correctly identify them as not having the disease 95% of the time.
Now, suppose that the disease is rare, occurring in only 1% of the population. If a person takes the test and tests positive for the disease, what is the probability that they actually have the disease?
Using Bayes’ Theorem, we can calculate the probability as follows:
- Let A be the event that the person has the disease, and B be the event that the test is positive.
- The prior probability of A is 0.01 (since the disease occurs in 1% of the population).
- The prior probability of B given A is 0.95 (since the test has a 95% accuracy rate).
- The prior probability of B given not A is 0.05 (since the test has a 5% false positive rate).
- The prior probability of not A is 0.99 (since the disease occurs in only 1% of the population, the probability of not having the disease is 0.99).
Using Bayes’ Theorem, we can calculate the probability of A given B as:
P(A|B) = P(B|A) * P(A) / P(B)
= 0.95 * 0.01 / ((0.95 * 0.01) + (0.05 * 0.99))
= 0.161
Bayes Theorem Formula
Bayes’ theorem can be proved using basic rules of probability. Let’s start with the definition of conditional probability:
P(A | B) = P(A ∩ B) / P(B)
where P(A ∩ B) is the probability of both events A and B occurring together.
We can also write this as:
P(B | A) = P(A ∩ B) / P(A)
where P(B | A) is the probability of event B occurring given that event A has occurred.
By setting these two expressions equal to each other, we get:
P(A ∩ B) / P(B) = P(A ∩ B) / P(A)
Now, if we multiply both sides of the equation by P(B), we get:
P(A ∩ B) = P(B | A) * P(A)
This is the numerator of Bayes’ theorem.
We can also use the law of total probability to express P(B) in terms of P(B | A) and P(B | not A):
P(B) = P(B | A) * P(A) + P(B | not A) * P(not A)
where P(not A) is the complement of event A.
Substituting this expression into the denominator of Bayes’ theorem, we get:
P(A | B) = P(B | A) * P(A) / (P(B | A) * P(A) + P(B | not A) * P(not A))
This is the full expression of Bayes’ theorem.
So, Bayes’ theorem follows from the definition of conditional probability and the law of total probability.
Examples and Solutions
Example 1: Suppose that a medical test for a certain disease is known to have a false positive rate of 2% and a false negative rate of 1%. If 1% of the population actually has the disease, what is the probability that a person who tests positive actually has the disease?
Solution: Let A be the event that a person has the disease, and let B be the event that the person tests positive. We want to find P(A | B), the probability that a person has the disease given that they tested positive.
Using Bayes’ theorem, we have:
P(A | B) = P(B | A) * P(A) / P(B)
where:
- P(B | A) is the probability of testing positive given that a person has the disease, which is (1 – 0.01) = 0.99 (the complement of the false negative rate).
- P(A) is the prior probability of a person having the disease, which is 0.01.
- P(B) is the total probability of testing positive, which can be calculated using the law of total probability as follows:
P(B) = P(B | A) * P(A) + P(B | not A) * P(not A)
= 0.99 * 0.01 + 0.02 * 0.99
= 0.0297
Therefore, we can calculate P(A | B) as:
P(A | B) = 0.99 * 0.01 / 0.0297 = 0.3333 or 33.33%
Example 2: Suppose that a company produces two types of products: 60% are Type A and 40% are Type B. Of the Type A products, 5% are defective, while of the Type B products, 10% are defective. A customer reports that their product is defective. What is the probability that it is a Type A product?
Solution: Let A be the event that the product is Type A, and let B be the event that the product is defective. We want to find P(A | B), the probability that the product is Type A given that it is defective.
Using Bayes’ theorem, we have:
P(A | B) = P(B | A) * P(A) / P(B)
where:
- P(B | A) is the probability of the product being defective given that it is Type A, which is 0.05.
- P(A) is the prior probability of the product being Type A, which is 0.6.
- P(B) is the total probability of the product being defective, which can be calculated using the law of total probability as follows:
P(B) = P(B | A) * P(A) + P(B | not A) * P(not A) = 0.05 * 0.6 + 0.1 * 0.4 = 0.07
Therefore, we can calculate P(A | B) as:
P(A | B) = 0.05 * 0.6 / 0.07 = 0.4286 or 42.86%