Difference between revisions of "Odds"

From Lesswrongwiki
Jump to: navigation, search
m (AstraSequi moved page Odds ratio to Odds: correct terminology)
 
(3 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Odds ratios are an alternate way of expressing probabilities, which simplifies the process of updating them with new evidence. The odds ratio of A is P(A)/P(¬A).
+
{{arbitallink|https://arbital.com/p/62c/|Bayes' Rule: Odds form}}Odds are an alternate way of expressing probabilities, which simplifies the process of updating them with new evidence. The odds of A is P(A)/P(¬A). The ratio of two odds is called an odds ratio, and the ratio of two probabilities is called a likelihood ratio. Every probability maps to an odds and vice versa, but in some circumstances odds are more useful due to their mathematical properties. While probabilities have a range of 0 to 1, odds have a range of 0 to ∞.
 +
 
 +
==Odds form of Bayes' Rule==
 +
 
 +
Bayes' Rule can be expressed in terms of odds: <math>Posterior ~ odds = Prior ~ odds \times Likelihood ~ ratio</math>.
 +
 
 +
The likelihood ratio is the relative probability of B being observed if hypothesis A is true, versus B being observed if hypothesis ¬A is true. Therefore, a Bayesian update can be calculated by converting the prior probability to odds, multiplying by the likelihood ratio, and converting the posterior odds back to probability. Knowing the probabilities for observing the evidence is unnecessary, only how many times more likely it is under one hypothesis than the other.
 +
 
 +
If the likelihood ratio is known, Bayesian updates are faster and more intuitive to calculate using the odds form. For example, if you know that A being true makes the observation of B twice as likely as when ¬A is true, the update can be calculated by converting the prior to odds, multiplying by two, and converting back. Additionally, if the prior is low, probability and odds can be approximated as each other (p=0.1 iff odds=0.111, and p=0.01 iff odds=0.0101), so the posterior probability can be approximated by skipping the conversion and simply multiplying by two.
 +
 
 +
==Proof of equivalence of the two forms==
  
 
<math>P(A|B) = P(B|A)\frac{P(A)}{P(B)}</math>
 
<math>P(A|B) = P(B|A)\frac{P(A)}{P(B)}</math>
Line 7: Line 17:
 
<math>\frac{P(A|B)}{P(\neg A|B)} = \frac{P(B|A)}{P(B|\neg A)}\frac{P(A)}{P(\neg A)}</math>
 
<math>\frac{P(A|B)}{P(\neg A|B)} = \frac{P(B|A)}{P(B|\neg A)}\frac{P(A)}{P(\neg A)}</math>
  
Thus, in order to find the posterior odds ratio <math>\frac{P(A|B)}{P(\neg A|B)}</math>, one simply multiplies the prior odds ratio <math>\frac{P(A)}{P(\neg A)}</math> by the likelihood ratio <math>\frac{P(B|A)}{P(B|\neg A)}</math>.
+
Thus, in order to find the posterior odds <math>\frac{P(A|B)}{P(\neg A|B)}</math>, one simply multiplies the prior odds <math>\frac{P(A)}{P(\neg A)}</math> by the likelihood ratio <math>\frac{P(B|A)}{P(B|\neg A)}</math>.  
 +
 
 +
==Odds, a:b, and probability==
  
Odds ratios are commonly written as the ratio of two numbers separated by a colon. For example, if P(A) = 2/3, the odds ratio would be 2, but this would most likely be written as 2:1.
+
Odds are commonly written as the ratio of two numbers separated by a colon. For example, if P(A) = 2/3, the odds would be 2, but this would most likely be written as 2:1.
  
The relation between odds ratio, a:b, and probability, p is as follows:
+
The relation between odds, a:b, and probability, p is as follows:
  
 
<math>a:b = p:(1-p)</math>
 
<math>a:b = p:(1-p)</math>
Line 17: Line 29:
 
<math>p = \frac{a}{a+b}</math>
 
<math>p = \frac{a}{a+b}</math>
  
Suppose you have a box that has a 5% chance of containing a diamond. You also have a diamond detector that beeps two thirds of the time if there is a diamond, and one third of the time if there is not. You wave the diamond detector over the box and it beeps.
+
Suppose you have a box that has a 5% chance of containing a diamond. You also have a diamond detector that beeps half of the time if there is a diamond, and one fourth of the time if there is not. You wave the diamond detector over the box and it beeps.
  
The prior odds of the box containing a diamond are 1:19. The likelihood ratio of a beep is 2/3:1/3 = 2:1. The posterior odds are 1:19 * 2:1 = 2:19. This corresponds to about a probability of 2/21, which is about 0.095 or 9.5%.
+
The prior odds of the box containing a diamond are 1:19. The likelihood ratio of a beep is 1/2:1/4 = 2:1. The posterior odds are 1:19 * 2:1 = 2:19. This corresponds to about a probability of 2/21, which is about 0.095 or 9.5%.
  
 
==See also==
 
==See also==

Latest revision as of 18:24, 7 February 2020

Arbital has an article about

Odds are an alternate way of expressing probabilities, which simplifies the process of updating them with new evidence. The odds of A is P(A)/P(¬A). The ratio of two odds is called an odds ratio, and the ratio of two probabilities is called a likelihood ratio. Every probability maps to an odds and vice versa, but in some circumstances odds are more useful due to their mathematical properties. While probabilities have a range of 0 to 1, odds have a range of 0 to ∞.

Odds form of Bayes' Rule

Bayes' Rule can be expressed in terms of odds: .

The likelihood ratio is the relative probability of B being observed if hypothesis A is true, versus B being observed if hypothesis ¬A is true. Therefore, a Bayesian update can be calculated by converting the prior probability to odds, multiplying by the likelihood ratio, and converting the posterior odds back to probability. Knowing the probabilities for observing the evidence is unnecessary, only how many times more likely it is under one hypothesis than the other.

If the likelihood ratio is known, Bayesian updates are faster and more intuitive to calculate using the odds form. For example, if you know that A being true makes the observation of B twice as likely as when ¬A is true, the update can be calculated by converting the prior to odds, multiplying by two, and converting back. Additionally, if the prior is low, probability and odds can be approximated as each other (p=0.1 iff odds=0.111, and p=0.01 iff odds=0.0101), so the posterior probability can be approximated by skipping the conversion and simply multiplying by two.

Proof of equivalence of the two forms

Thus, in order to find the posterior odds , one simply multiplies the prior odds by the likelihood ratio .

Odds, a:b, and probability

Odds are commonly written as the ratio of two numbers separated by a colon. For example, if P(A) = 2/3, the odds would be 2, but this would most likely be written as 2:1.

The relation between odds, a:b, and probability, p is as follows:

Suppose you have a box that has a 5% chance of containing a diamond. You also have a diamond detector that beeps half of the time if there is a diamond, and one fourth of the time if there is not. You wave the diamond detector over the box and it beeps.

The prior odds of the box containing a diamond are 1:19. The likelihood ratio of a beep is 1/2:1/4 = 2:1. The posterior odds are 1:19 * 2:1 = 2:19. This corresponds to about a probability of 2/21, which is about 0.095 or 9.5%.

See also