# Difference between revisions of "Bayesian"

Steven0461 (talk | contribs) m |
|||

(16 intermediate revisions by 6 users not shown) | |||

Line 1: | Line 1: | ||

{{wikilink|Bayesian}} | {{wikilink|Bayesian}} | ||

− | + | {{arbitallink|https://arbital.com/p/bayes_rule_guide/|Bayes Rule Guide}} | |

The secret technical codeword that cognitive scientists use to mean "rational". Bayesian [[probability theory]] is the math of [[epistemic rationality]], Bayesian [[decision theory]] is the math of [[instrumental rationality]]. Right up there with [[cognitive bias]] as an absolutely fundamental concept on Less Wrong. | The secret technical codeword that cognitive scientists use to mean "rational". Bayesian [[probability theory]] is the math of [[epistemic rationality]], Bayesian [[decision theory]] is the math of [[instrumental rationality]]. Right up there with [[cognitive bias]] as an absolutely fundamental concept on Less Wrong. | ||

==Philosophy== | ==Philosophy== | ||

− | Classical statistics is a bucket of assorted methods; different "methods" may give different answers for whether, e.g., an experimental result is "statistically significant". In contrast, as the famous Bayesian E. T. Jaynes emphasized, probability theory is ''math'' and its results are ''theorems'', every theorem consistent with every other theorem; you cannot get two different results by doing the derivation two different ways. | + | [[Classical statistics]] is a bucket of assorted methods; different "methods" may give different answers for whether, e.g., an experimental result is "statistically significant". In contrast, as the famous Bayesian E. T. Jaynes emphasized, probability theory is ''math'' and its results are ''theorems'', every theorem consistent with every other theorem; you cannot get two different results by doing the derivation two different ways. |

So is the project of rationality solved? Indeed not. First, probability theory and decision theory are often too computationally expensive to run in practice - it wouldn't take a galaxy-sized computer, so much as an unphysical computer (much larger than the known universe). And second, it's not always clear how the math ''applies'' - even in theory, let alone the practice. | So is the project of rationality solved? Indeed not. First, probability theory and decision theory are often too computationally expensive to run in practice - it wouldn't take a galaxy-sized computer, so much as an unphysical computer (much larger than the known universe). And second, it's not always clear how the math ''applies'' - even in theory, let alone the practice. | ||

− | But we do know that violations of Bayesianism - even "unavoidable" violations due to lack of computing power - carry | + | But we do know that violations of Bayesianism - even "unavoidable" violations due to lack of computing power - carry a price; a family of theorems demonstrates that anyone who does not choose according to consistent probabilities can be made to accept combinations of bets that are sure losses, or reject bets that are sure wins (the [[Dutch Book]] arguments); similarly, [[Cox's Theorem]] and its extensions show that anyone who obeys various "common-sensical" constraints on their betting probabilities must be representable in standard probability theory. |

In other words, Bayesianism isn't just a good idea - ''it's the law'', and if you violate it, you'll pay ''some'' kind of price. | In other words, Bayesianism isn't just a good idea - ''it's the law'', and if you violate it, you'll pay ''some'' kind of price. | ||

Line 17: | Line 17: | ||

==Math== | ==Math== | ||

− | (Needs to be fleshed out.) For introductions see [[probability theory]] | + | (Needs to be fleshed out.) For introductions see [[probability theory]], [[decision theory]], and [http://yudkowsky.net/rational/bayes this introduction] to [[Bayes' theorem]]. A widely lauded technical book on this subject is E. T. Jaynes's [http://www-biba.inrialpes.fr/Jaynes/prob.html "Probability Theory: The Logic of Science"]. |

==Other usages== | ==Other usages== | ||

Line 25: | Line 25: | ||

The term "Bayesian" may also refer to an ideal rational agent implementing precise, perfect Bayesian probability theory and decision theory (see, for example, [[Aumann's agreement theorem]]). | The term "Bayesian" may also refer to an ideal rational agent implementing precise, perfect Bayesian probability theory and decision theory (see, for example, [[Aumann's agreement theorem]]). | ||

− | + | ==Blog posts== | |

− | |||

− | |||

− | |||

− | |||

+ | *{{lesswrongtag}} | ||

+ | *[http://lesswrong.com/lw/1to/what_is_bayesianism/ What is Bayesianism] | ||

*[http://lesswrong.com/lw/mt/beautiful_probability/ Beautiful Probability], [http://lesswrong.com/lw/mu/trust_in_math/ Trust in Math], and [http://lesswrong.com/lw/na/trust_in_bayes/ Trust in Bayes] | *[http://lesswrong.com/lw/mt/beautiful_probability/ Beautiful Probability], [http://lesswrong.com/lw/mu/trust_in_math/ Trust in Math], and [http://lesswrong.com/lw/na/trust_in_bayes/ Trust in Bayes] | ||

− | *[http://lesswrong.com/lw/oj/probability_is_in_the_mind/ Probability is in the | + | *[http://lesswrong.com/lw/oj/probability_is_in_the_mind/ Probability is in the Mind], [http://lesswrong.com/lw/s6/probability_is_subjectively_objective/ Probability is Subjectively Objective], and [http://lesswrong.com/lw/om/qualitatively_confused/ Qualitatively Confused] |

− | *[http://lesswrong.com/lw/o5/the_second_law_of_thermodynamics_and_engines_of/ The Second Law of Thermodynamics, and Engines of Cognition], [http://lesswrong.com/lw/o6/perpetual_motion_beliefs/ Perpetual Motion Beliefs] and [http://lesswrong.com/lw/o7/searching_for_bayesstructure/ Searching for Bayes- | + | *[http://lesswrong.com/lw/o5/the_second_law_of_thermodynamics_and_engines_of/ The Second Law of Thermodynamics, and Engines of Cognition], [http://lesswrong.com/lw/o6/perpetual_motion_beliefs/ Perpetual Motion Beliefs] and [http://lesswrong.com/lw/o7/searching_for_bayesstructure/ Searching for Bayes-Structure] |

*[http://lesswrong.com/lw/ul/my_bayesian_enlightenment/ My Bayesian Enlightenment] | *[http://lesswrong.com/lw/ul/my_bayesian_enlightenment/ My Bayesian Enlightenment] | ||

*[http://lesswrong.com/lw/k2/a_priori/ A Priori] | *[http://lesswrong.com/lw/k2/a_priori/ A Priori] | ||

*[http://lesswrong.com/lw/hk/priors_as_mathematical_objects/ Priors as Mathematical Objects] | *[http://lesswrong.com/lw/hk/priors_as_mathematical_objects/ Priors as Mathematical Objects] | ||

− | *[http://lesswrong.com/lw/k1/no_one_can_exempt_you_from_rationalitys_laws/ No | + | *[http://lesswrong.com/lw/k1/no_one_can_exempt_you_from_rationalitys_laws/ No One Can Exempt You From Rationality's Laws] |

*[http://lesswrong.com/lw/l4/terminal_values_and_instrumental_values/ Terminal Values and Instrumental Values] | *[http://lesswrong.com/lw/l4/terminal_values_and_instrumental_values/ Terminal Values and Instrumental Values] | ||

− | *[http://lesswrong.com/lw/vo/lawful_uncertainty/ Lawful | + | *[http://lesswrong.com/lw/vo/lawful_uncertainty/ Lawful Uncertainty] |

− | *[http://lesswrong.com/lw/n3/circular_altruism/ Circular | + | *[http://lesswrong.com/lw/n3/circular_altruism/ Circular Altruism] |

*[http://lesswrong.com/lw/nc/newcombs_problem_and_regret_of_rationality/ Newcomb's Problem and Regret of Rationality] | *[http://lesswrong.com/lw/nc/newcombs_problem_and_regret_of_rationality/ Newcomb's Problem and Regret of Rationality] | ||

− | *[http://lesswrong.com/lw/sg/when_not_to_use_probabilities/ When ( | + | *[http://lesswrong.com/lw/sg/when_not_to_use_probabilities/ When (Not) To Use Probabilities] |

*[http://lesswrong.com/lw/qk/that_alien_message/ That Alien Message] | *[http://lesswrong.com/lw/qk/that_alien_message/ That Alien Message] | ||

− | *[http://lesswrong.com/lw/qg/changing_the_definition_of_science/ Changing the | + | *[http://lesswrong.com/lw/qg/changing_the_definition_of_science/ Changing the Definition of Science] |

==See also== | ==See also== | ||

− | + | ||

*[[Bayes theorem]] | *[[Bayes theorem]] | ||

*[[Bayesian probability]] | *[[Bayesian probability]] | ||

+ | *[[Priors]] | ||

*[[Rational evidence]] | *[[Rational evidence]] | ||

*[[Probability theory]] | *[[Probability theory]] | ||

*[[Decision theory]] | *[[Decision theory]] | ||

+ | *[[Lawful intelligence]] | ||

*[[Bayesian Conspiracy]] | *[[Bayesian Conspiracy]] | ||

− | |||

[[Category:Concepts]] | [[Category:Concepts]] | ||

[[Category:Jargon]] | [[Category:Jargon]] | ||

+ | [[Category:Bayesian]] | ||

+ | |||

+ | {{cleanup}} |

## Latest revision as of 05:58, 28 August 2016

The secret technical codeword that cognitive scientists use to mean "rational". Bayesian probability theory is the math of epistemic rationality, Bayesian decision theory is the math of instrumental rationality. Right up there with cognitive bias as an absolutely fundamental concept on Less Wrong.

## Philosophy

Classical statistics is a bucket of assorted methods; different "methods" may give different answers for whether, e.g., an experimental result is "statistically significant". In contrast, as the famous Bayesian E. T. Jaynes emphasized, probability theory is *math* and its results are *theorems*, every theorem consistent with every other theorem; you cannot get two different results by doing the derivation two different ways.

So is the project of rationality solved? Indeed not. First, probability theory and decision theory are often too computationally expensive to run in practice - it wouldn't take a galaxy-sized computer, so much as an unphysical computer (much larger than the known universe). And second, it's not always clear how the math *applies* - even in theory, let alone the practice.

But we do know that violations of Bayesianism - even "unavoidable" violations due to lack of computing power - carry a price; a family of theorems demonstrates that anyone who does not choose according to consistent probabilities can be made to accept combinations of bets that are sure losses, or reject bets that are sure wins (the Dutch Book arguments); similarly, Cox's Theorem and its extensions show that anyone who obeys various "common-sensical" constraints on their betting probabilities must be representable in standard probability theory.

In other words, Bayesianism isn't just a good idea - *it's the law*, and if you violate it, you'll pay *some* kind of price.

When cognitive psychologists identify a cognitive bias, they know it's an *error* by comparison to the Bayesian gold standard.

## Math

(Needs to be fleshed out.) For introductions see probability theory, decision theory, and this introduction to Bayes' theorem. A widely lauded technical book on this subject is E. T. Jaynes's "Probability Theory: The Logic of Science".

## Other usages

"Bayesian" in philosophical usage often describes someone who adheres to the Bayesian interpretation of probability, viewing probability as a level of certainty in a potential outcome or idea. This is in contrast to a frequentist who views probability as a representation of how frequently a particular outcome will occur over any number of trials.

The term "Bayesian" may also refer to an ideal rational agent implementing precise, perfect Bayesian probability theory and decision theory (see, for example, Aumann's agreement theorem).

## Blog posts

- All Less Wrong posts tagged "Bayesian"
- What is Bayesianism
- Beautiful Probability, Trust in Math, and Trust in Bayes
- Probability is in the Mind, Probability is Subjectively Objective, and Qualitatively Confused
- The Second Law of Thermodynamics, and Engines of Cognition, Perpetual Motion Beliefs and Searching for Bayes-Structure
- My Bayesian Enlightenment
- A Priori
- Priors as Mathematical Objects

- No One Can Exempt You From Rationality's Laws
- Terminal Values and Instrumental Values
- Lawful Uncertainty
- Circular Altruism
- Newcomb's Problem and Regret of Rationality
- When (Not) To Use Probabilities
- That Alien Message
- Changing the Definition of Science

## See also

- Bayes theorem
- Bayesian probability
- Priors
- Rational evidence
- Probability theory
- Decision theory
- Lawful intelligence
- Bayesian Conspiracy