# Difference between revisions of "Bayesian"

(Bayesian moved to Bayesian probability: Bayesian, as used on OB/LW, is an agent implementing Bayesian inference. This article is about Bayesian interpretation of probability.) |
|||

(30 intermediate revisions by 8 users not shown) | |||

Line 1: | Line 1: | ||

− | + | {{wikilink|Bayesian}} | |

+ | {{arbitallink|https://arbital.com/p/bayes_rule_guide/|Bayes Rule Guide}} | ||

+ | The secret technical codeword that cognitive scientists use to mean "rational". Bayesian [[probability theory]] is the math of [[epistemic rationality]], Bayesian [[decision theory]] is the math of [[instrumental rationality]]. Right up there with [[cognitive bias]] as an absolutely fundamental concept on Less Wrong. | ||

+ | |||

+ | ==Philosophy== | ||

+ | |||

+ | [[Classical statistics]] is a bucket of assorted methods; different "methods" may give different answers for whether, e.g., an experimental result is "statistically significant". In contrast, as the famous Bayesian E. T. Jaynes emphasized, probability theory is ''math'' and its results are ''theorems'', every theorem consistent with every other theorem; you cannot get two different results by doing the derivation two different ways. | ||

+ | |||

+ | So is the project of rationality solved? Indeed not. First, probability theory and decision theory are often too computationally expensive to run in practice - it wouldn't take a galaxy-sized computer, so much as an unphysical computer (much larger than the known universe). And second, it's not always clear how the math ''applies'' - even in theory, let alone the practice. | ||

+ | |||

+ | But we do know that violations of Bayesianism - even "unavoidable" violations due to lack of computing power - carry a price; a family of theorems demonstrates that anyone who does not choose according to consistent probabilities can be made to accept combinations of bets that are sure losses, or reject bets that are sure wins (the [[Dutch Book]] arguments); similarly, [[Cox's Theorem]] and its extensions show that anyone who obeys various "common-sensical" constraints on their betting probabilities must be representable in standard probability theory. | ||

+ | |||

+ | In other words, Bayesianism isn't just a good idea - ''it's the law'', and if you violate it, you'll pay ''some'' kind of price. | ||

+ | |||

+ | When cognitive psychologists identify a [[cognitive bias]], they know it's an ''error'' by comparison to the Bayesian gold standard. | ||

+ | |||

+ | ==Math== | ||

+ | |||

+ | (Needs to be fleshed out.) For introductions see [[probability theory]], [[decision theory]], and [http://yudkowsky.net/rational/bayes this introduction] to [[Bayes' theorem]]. A widely lauded technical book on this subject is E. T. Jaynes's [http://www-biba.inrialpes.fr/Jaynes/prob.html "Probability Theory: The Logic of Science"]. | ||

+ | |||

+ | ==Other usages== | ||

+ | |||

+ | "Bayesian" in philosophical usage often describes someone who adheres to the [[Bayesian probability|Bayesian interpretation]] of probability, viewing probability as a level of certainty in a potential outcome or idea. This is in contrast to a [[frequentist]] who views probability as a representation of how frequently a particular outcome will occur over any number of trials. | ||

+ | |||

+ | The term "Bayesian" may also refer to an ideal rational agent implementing precise, perfect Bayesian probability theory and decision theory (see, for example, [[Aumann's agreement theorem]]). | ||

+ | |||

+ | ==Blog posts== | ||

+ | |||

+ | *{{lesswrongtag}} | ||

+ | *[http://lesswrong.com/lw/1to/what_is_bayesianism/ What is Bayesianism] | ||

+ | *[http://lesswrong.com/lw/mt/beautiful_probability/ Beautiful Probability], [http://lesswrong.com/lw/mu/trust_in_math/ Trust in Math], and [http://lesswrong.com/lw/na/trust_in_bayes/ Trust in Bayes] | ||

+ | *[http://lesswrong.com/lw/oj/probability_is_in_the_mind/ Probability is in the Mind], [http://lesswrong.com/lw/s6/probability_is_subjectively_objective/ Probability is Subjectively Objective], and [http://lesswrong.com/lw/om/qualitatively_confused/ Qualitatively Confused] | ||

+ | *[http://lesswrong.com/lw/o5/the_second_law_of_thermodynamics_and_engines_of/ The Second Law of Thermodynamics, and Engines of Cognition], [http://lesswrong.com/lw/o6/perpetual_motion_beliefs/ Perpetual Motion Beliefs] and [http://lesswrong.com/lw/o7/searching_for_bayesstructure/ Searching for Bayes-Structure] | ||

+ | *[http://lesswrong.com/lw/ul/my_bayesian_enlightenment/ My Bayesian Enlightenment] | ||

+ | *[http://lesswrong.com/lw/k2/a_priori/ A Priori] | ||

+ | *[http://lesswrong.com/lw/hk/priors_as_mathematical_objects/ Priors as Mathematical Objects] | ||

+ | |||

+ | *[http://lesswrong.com/lw/k1/no_one_can_exempt_you_from_rationalitys_laws/ No One Can Exempt You From Rationality's Laws] | ||

+ | *[http://lesswrong.com/lw/l4/terminal_values_and_instrumental_values/ Terminal Values and Instrumental Values] | ||

+ | *[http://lesswrong.com/lw/vo/lawful_uncertainty/ Lawful Uncertainty] | ||

+ | *[http://lesswrong.com/lw/n3/circular_altruism/ Circular Altruism] | ||

+ | *[http://lesswrong.com/lw/nc/newcombs_problem_and_regret_of_rationality/ Newcomb's Problem and Regret of Rationality] | ||

+ | *[http://lesswrong.com/lw/sg/when_not_to_use_probabilities/ When (Not) To Use Probabilities] | ||

+ | *[http://lesswrong.com/lw/qk/that_alien_message/ That Alien Message] | ||

+ | *[http://lesswrong.com/lw/qg/changing_the_definition_of_science/ Changing the Definition of Science] | ||

+ | |||

+ | ==See also== | ||

+ | |||

+ | *[[Bayes theorem]] | ||

+ | *[[Bayesian probability]] | ||

+ | *[[Priors]] | ||

+ | *[[Rational evidence]] | ||

+ | *[[Probability theory]] | ||

+ | *[[Decision theory]] | ||

+ | *[[Lawful intelligence]] | ||

+ | *[[Bayesian Conspiracy]] | ||

+ | |||

+ | [[Category:Concepts]] | ||

+ | [[Category:Jargon]] | ||

+ | [[Category:Bayesian]] | ||

+ | |||

+ | {{cleanup}} |

## Latest revision as of 04:58, 28 August 2016

The secret technical codeword that cognitive scientists use to mean "rational". Bayesian probability theory is the math of epistemic rationality, Bayesian decision theory is the math of instrumental rationality. Right up there with cognitive bias as an absolutely fundamental concept on Less Wrong.

## Philosophy

Classical statistics is a bucket of assorted methods; different "methods" may give different answers for whether, e.g., an experimental result is "statistically significant". In contrast, as the famous Bayesian E. T. Jaynes emphasized, probability theory is *math* and its results are *theorems*, every theorem consistent with every other theorem; you cannot get two different results by doing the derivation two different ways.

So is the project of rationality solved? Indeed not. First, probability theory and decision theory are often too computationally expensive to run in practice - it wouldn't take a galaxy-sized computer, so much as an unphysical computer (much larger than the known universe). And second, it's not always clear how the math *applies* - even in theory, let alone the practice.

But we do know that violations of Bayesianism - even "unavoidable" violations due to lack of computing power - carry a price; a family of theorems demonstrates that anyone who does not choose according to consistent probabilities can be made to accept combinations of bets that are sure losses, or reject bets that are sure wins (the Dutch Book arguments); similarly, Cox's Theorem and its extensions show that anyone who obeys various "common-sensical" constraints on their betting probabilities must be representable in standard probability theory.

In other words, Bayesianism isn't just a good idea - *it's the law*, and if you violate it, you'll pay *some* kind of price.

When cognitive psychologists identify a cognitive bias, they know it's an *error* by comparison to the Bayesian gold standard.

## Math

(Needs to be fleshed out.) For introductions see probability theory, decision theory, and this introduction to Bayes' theorem. A widely lauded technical book on this subject is E. T. Jaynes's "Probability Theory: The Logic of Science".

## Other usages

"Bayesian" in philosophical usage often describes someone who adheres to the Bayesian interpretation of probability, viewing probability as a level of certainty in a potential outcome or idea. This is in contrast to a frequentist who views probability as a representation of how frequently a particular outcome will occur over any number of trials.

The term "Bayesian" may also refer to an ideal rational agent implementing precise, perfect Bayesian probability theory and decision theory (see, for example, Aumann's agreement theorem).

## Blog posts

- All Less Wrong posts tagged "Bayesian"
- What is Bayesianism
- Beautiful Probability, Trust in Math, and Trust in Bayes
- Probability is in the Mind, Probability is Subjectively Objective, and Qualitatively Confused
- The Second Law of Thermodynamics, and Engines of Cognition, Perpetual Motion Beliefs and Searching for Bayes-Structure
- My Bayesian Enlightenment
- A Priori
- Priors as Mathematical Objects

- No One Can Exempt You From Rationality's Laws
- Terminal Values and Instrumental Values
- Lawful Uncertainty
- Circular Altruism
- Newcomb's Problem and Regret of Rationality
- When (Not) To Use Probabilities
- That Alien Message
- Changing the Definition of Science

## See also

- Bayes theorem
- Bayesian probability
- Priors
- Rational evidence
- Probability theory
- Decision theory
- Lawful intelligence
- Bayesian Conspiracy