Bayes’ Theorem
It is a simple mathematical formula which is used for calculating conditional probabilities.
Bayesian networks, which is a type of probabilistic graphical model, is used to build models from acquired data, or mined opinions. This is used for a variety of tasks which include prediction, anomaly detection, diagnostics,automated insight, reasoning, time series prediction or decision making under uncertainty.
Capabilities in terms of the four major analytics disciplines are
Descriptive analytics, Diagnostic analytics, Predictive analytics and Prescriptive analytics.
Conditional Probabilities and Bayes' Theorem
Conditional probability can be calculated using the other conditional probability:
P(A|B) = P(B|A) * P(A) / P(B)
The reverse is also true:
P(B|A) = P(A|B) * P(B) / P(A)
This approach of calculating the conditional probability is useful when the joint probability is challenging to calculate, or when the reverse conditional probability is available or easy to calculate.
This alternate calculation of the conditional probability is known as Bayes Rule or Bayes Theorem.
Bayes Theorem is a Principled way of calculating a conditional probability without the joint probability.
Since we don’t have the access to the denominator directly, i.e. P(B).
We can calculate it another way:
P(B) = P(B|A) * P(A) + P(B|not A) * P(not A)
Calculation of P(B), described below:
P(A|B) = P(B|A) * P(A) / P(B|A) * P(A) + P(B|not A) * P(not A)
For clarity:
P(A|B) = P(B|A) * P(A) / (P(B|A) * P(A) + P(B|not A) * P(not A))
A real-life usage for the benefit of Bayes Theorem is in the analysis of a medical diagnostic test.
BAYES THEOREM AND BREAST CANCER
In the human population, some might have cancer and others might not with cancer is true or false, and a medical test for the same returns with either positive or negative. For detection of breast cancer, mammograms do the trick. Now if a randomly selected patient goes forward with the test and it comes back positive, what is the probability that the patient has cancer?
Anatomy of a Test
Considering a cancer testing scenario where
1% of women have breast cancer (leaving a population of 99% who do not).
80% of mammograms detect breast cancer when it is there (which misses 20% of it).
9.6% of mammograms detect breast cancer when it’s not there (Which means 90.4% correctly return a negative result).
Considering this, We read the table as:
1% of the total population have cancer
If you already have cancer, there’s an 80% chance you will test positive leaving a 20% chance of testing negative.
If you don’t have cancer, there’s a 9.6% chance you will test positive, leaving a 90.4% chance of testing negative.
Now in case of a positive test result, what could be the actual chances of having cancer?
The chances of a true positive = chance you have cancer * chance test caught it = 1% * 80% = .008
The chances of a false positive = chance you don’t have cancer * chance test caught it anyway = 99% * 9.6% = 0.09504
The chances of an event if the number of possible ways an outcome can occur.
Chance of getting a real, positive result is .008, But the chance of getting any type of positive result is the chance of a true positive and a chance of a false positive (.008 + 0.09504 = .10304) making chance of cancer .008/.10304 = 0.0776, or about 7.8%.
A positive mammogram only means one has a chance of 7.8% of having cancer, rather than an 80% (which is the supposed accuracy of the test). It seems strange but considering the test gives a false positive of 9.6% of the time (which is quite high), That makes a lot of false positives in a given population. For a rare disease, like breast cancer most of the positive test results will be wrong.
Now If we take 100 people, every/only 1 in 100 people will have cancer (1%), and are going to test positive (80% chance). But in the remaining 99 people, about 10% more will test positive, giving us a rough approximation of 10 false positives. Considering this, just 1 in 11 is correct, so there’s a 1/11 chance of having cancer given a positive test. The real number is 7.8% (closer to 1/13, computed above), which was close to the one we estimated. We can turn the whole estimation into an equation which is Bayes Theorem, it lets us get the real chance of having the event, i.e. finding the probability of having cancer.
Pr(H|E) = Chance of having cancer (H) given a positive test (E). Getting cancer with a positive result in our case is 7.8%.
Pr(E|H) = Chance of a positive test (E) given that you had cancer (H). This is the chance of a true positive, 80% in our case.
Pr(H) = Chance of having cancer (1%).
Pr(not H) = Chance of not having cancer (99%).
Pr(E|not H) = Chance of a positive test (E) given that you didn’t have cancer (not H). This is a false positive, 9.6% in our case.
Eventually, we need to find the chance of a true positive divided by the chance of any positive. The equation thus becomes-
Pr(E) gives us the chance of getting any positive result, whether a true positive in the cancer population (1%) or a false positive in the non-cancer population (99%). One of the reasons is forgetting to account for false positives gives us a low of 7.8% chance of cancer (given a positive test) which may seem counter-intuitive. Hence there we use a normalizing constant which increases the accuracy of the test. Bayes’ Theorem helps us look at the skewed test results and correct for errors, which recreates the original population therefore helps us in finding the real chance of a true positive result.