Difference between revisions of "Eliezer Yudkowsky"

From Lesswrongwiki
Jump to: navigation, search
 
(10 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 
{{wikilink}}
 
{{wikilink}}
{{afwikilink}}
+
 
'''Eliezer Yudkowsky''' is a research fellow of the [[Singularity Institute for Artificial Intelligence]] - which he co-founded in 2001. He is mainly concerned with the obstacles and importance of developing a [[Friendly AI]], such as a reflective decision theory that would lay a foundation for describing fully recursive self modifying agents that retain stable preferences while rewriting their source code.  He also co-founded Less Wrong, writing most part of The [[Sequences]], long sequences of posts dealing with epistemology, [[AGI]], [[metaethics]], [[rationality]] and so on.
+
'''Eliezer Yudkowsky''' is a research fellow of the [[Machine Intelligence Research Institute]], which he co-founded in 2001. He is mainly concerned with the obstacles and importance of developing a [[Friendly AI]], such as a reflective decision theory that would lay a foundation for describing fully recursive self modifying agents that retain stable preferences while rewriting their source code.  He also co-founded ''Less Wrong'', writing the [[Sequences]], long sequences of posts dealing with epistemology, [[AGI]], [[metaethics]], [[rationality]] and so on.
  
 
He has published several articles, including:   
 
He has published several articles, including:   
*[http://intelligence.org/files/CognitiveBiases.pdf “Cognitive Biases Potentially Affecting Judgment of Global Risks” (2008)]: A pioneer compilation of [[Bias|Cognitive Biases]] – systematic deviations from rationality – influencing our judgment of Global Catastrophic Risks. These are defined as risks with the potential to inflict serious damage to human well-being on a global scale, threatening more than millions of human lives(e.g.: volcanic eruptions, pandemic infections, nuclear accidents, worldwide tyrannies, out-of-control scientific experiments, climatic changes, cosmic hazards and economic collapse). It is a book chapter from a larger tome analyzing those risks.  
+
*[http://intelligence.org/files/CognitiveBiases.pdf “Cognitive Biases Potentially Affecting Judgment of Global Risks” (2008)]: A pioneer compilation of [[Bias|cognitive biases]] – systematic deviations from rationality – influencing our judgment of global catastrophic risks. These are defined as ''"where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential."'' Examples include volcanic eruptions, pandemic infections, nuclear war, worldwide tyrannies, out-of-control scientific experiments, or cosmic hazards). Yudkowsky's chapter specifically examines how cognitive biases impact thinking about global catastrophic risks.
*[http://intelligence.org/files/AIRisk.pdf “AI as a Positive and Negative Factor in Global Risk. (2008)”]: Chapter of the same book of the previous paper, it analyses possible philosophical and technical failures in the construction of a [[Friendly AI]], which could lead to an [[Unfriendily AI]] posing a enormous global risk. He also discusses how a Friendly AI could help decrease some Global Risks discussed in the book. Finally, because a powerful AI could go from been a Global Risk to help reduce some of them, he argues that researching such topic is extremely important.
+
*[http://intelligence.org/files/AIRisk.pdf “AI as a Positive and Negative Factor in Global Risk. (2008)”]: Another chapter in the compilation "Global Catastrophic Risks", it analyses possible philosophical and technical failures in the construction of a [[Friendly AI]], which could lead to an [[Unfriendly AI]] posing a enormous global risk. He also discusses how a Friendly AI could help decrease some global risks discussed in the book. Finally, because a powerful Friendly AI could reduce global risk, he argues that researching Friendly AI is extremely important for the future of humanity.
*[http://intelligence.org/upload/CFAI/index.html "Creating Friendly AI"(2001)]: One of the first articles to address the challenges in designing the features and cognitive architecture required to produce a benevolent - "Friendly" - Artificial Intelligence . It also gives one of the first precise definitions of terms such as Friendly AI and [[Seed AI]].
+
*[http://intelligence.org/files/CFAI.pdf "Creating Friendly AI"(2001)]: One of the first articles to address the challenges in designing the features and cognitive architecture required to produce a benevolent "Friendly" Artificial Intelligence . It also gives one of the first precise definitions of terms such as Friendly AI and [[Seed AI]].
*[http://intelligence.org/upload/LOGI//LOGI.pdf "Levels of Organization in General Intelligence" (2002)]: Analyses [[AGI]] through its decomposition in five subsystems, successive levels of functional organization:  Code, sensory modalities, concepts, thoughts, and deliberation. It also discusses some advantages artificial minds would have, such as the possibility of [[Recursive self-improvement]].
+
*[http://intelligence.org/files/LOGI.pdf "Levels of Organization in General Intelligence" (2002)]: Analyzes [[AGI]] through its decomposition in five subsystems, successive levels of functional organization:  Code, sensory modalities, concepts, thoughts, and deliberation. Also discusses some advantages artificial minds would have, such as the possibility of [[Recursive self-improvement]].
*[http://intelligence.org/upload/CEV.html "Coherent Extrapolated Volition"(2004)]: Presents the difficulties and possible solutions for incorporating friendliness into an AGI. It proposes that making an AGI doing what we tell it to could be dangerous, since we don`t know what we want. Instead we should program the AGI to do what we want, predicting what the vectorial sum of an idealized version of us would want, "if we knew more, thought faster, were more the people we wished we were, had grown up farther together”. He calls this the coherent extrapolated volition of humankind, or [[CEV]].
+
*[http://intelligence.org/files/CEV.html "Coherent Extrapolated Volition"(2004)]: Presents the difficulties and possible solutions for incorporating friendliness into an AGI. It argues that making an AGI simply do what we tell it to could be dangerous, since we don't know what we want. Instead we should program the AGI to do what we want, predicting what the vectorial sum of an idealized version of what humanity would want, ''"if we knew more, thought faster, were more the people we wished we were, had grown up farther together”''. He calls this the coherent extrapolated volition of humankind, or [[CEV]].
*[http://intelligence.org/upload/TDT-v01o.pdf "Timeless Decision Theory" (2010)]: Describes [[Timeless decision theory]],” an extension of causal decision networks that compactly represents uncertainty about correlated computational processes and represents the decision maker as such a process”. It solves many problems which [[Causal Decision Theory]] doesn`t have a plausible solution: [[Newcomb’s Problem]], Solomon’s Problems and [[Prisoner’s Dilemma]].
+
*[http://intelligence.org/files/TDT.pdf "Timeless Decision Theory" (2010)]: Describes [[Timeless decision theory]], ''”an extension of causal decision networks that compactly represents uncertainty about correlated computational processes and represents the decision maker as such a process”''. It solves many problems which [[Causal Decision Theory]] or [[Evidential Decision Theory]] don't have a plausible solution: [[Newcomb's problem]], [[Smoking lesion|Solomon's Problems]] and [[Prisoner's dilemma]].
*[http://intelligence.org/upload/complex-value-systems.pdf"Complex Value Systems are Required to Realize Valuable Futures" (2011)]: Discusses the [[Complexity of values]]: we can’t come up with a simple rule or description that sums up all human values. It analyses how this problem makes it difficult to build a valuable future.
+
*[http://intelligence.org/files/ComplexValues.pdf "Complex Value Systems are Required to Realize Valuable Futures" (2011)]: Discusses the [[Complexity of value]]: we can’t come up with a simple rule or description that sums up all human values. It analyzes how this problem makes it difficult to build a valuable future.
  
 
==Links==
 
==Links==
  
*[http://lesswrong.com/user/Eliezer_Yudkowsky Eliezer Yudkowsky's user page] at [http://lesswrong.com/ Less Wrong]
+
*[http://lesswrong.com/user/Eliezer_Yudkowsky/submitted/ Eliezer Yudkowsky's posts] on [http://lesswrong.com/ Less Wrong]
 
*[http://www.cs.auckland.ac.nz/~andwhay/postlist.html A list of all of Yudkowsky's posts to Overcoming Bias], [http://www.cs.auckland.ac.nz/~andwhay/graphlist.html Dependency graphs for them]
 
*[http://www.cs.auckland.ac.nz/~andwhay/postlist.html A list of all of Yudkowsky's posts to Overcoming Bias], [http://www.cs.auckland.ac.nz/~andwhay/graphlist.html Dependency graphs for them]
 
*[http://lesswrong.com/lw/4g/eliezer_yudkowsky_facts/ Eliezer Yudkowsky Facts] by [[steven0461]]
 
*[http://lesswrong.com/lw/4g/eliezer_yudkowsky_facts/ Eliezer Yudkowsky Facts] by [[steven0461]]
Line 21: Line 21:
 
[[Category:People]]
 
[[Category:People]]
 
[[Category:Authors]]
 
[[Category:Authors]]
 +
[[Category:Eliezer Yudkowsky]]

Latest revision as of 23:06, 7 May 2017

Smallwikipedialogo.png
Wikipedia has an article about


Eliezer Yudkowsky is a research fellow of the Machine Intelligence Research Institute, which he co-founded in 2001. He is mainly concerned with the obstacles and importance of developing a Friendly AI, such as a reflective decision theory that would lay a foundation for describing fully recursive self modifying agents that retain stable preferences while rewriting their source code. He also co-founded Less Wrong, writing the Sequences, long sequences of posts dealing with epistemology, AGI, metaethics, rationality and so on.

He has published several articles, including:

  • “Cognitive Biases Potentially Affecting Judgment of Global Risks” (2008): A pioneer compilation of cognitive biases – systematic deviations from rationality – influencing our judgment of global catastrophic risks. These are defined as "where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential." Examples include volcanic eruptions, pandemic infections, nuclear war, worldwide tyrannies, out-of-control scientific experiments, or cosmic hazards). Yudkowsky's chapter specifically examines how cognitive biases impact thinking about global catastrophic risks.
  • “AI as a Positive and Negative Factor in Global Risk. (2008)”: Another chapter in the compilation "Global Catastrophic Risks", it analyses possible philosophical and technical failures in the construction of a Friendly AI, which could lead to an Unfriendly AI posing a enormous global risk. He also discusses how a Friendly AI could help decrease some global risks discussed in the book. Finally, because a powerful Friendly AI could reduce global risk, he argues that researching Friendly AI is extremely important for the future of humanity.
  • "Creating Friendly AI"(2001): One of the first articles to address the challenges in designing the features and cognitive architecture required to produce a benevolent — "Friendly" — Artificial Intelligence . It also gives one of the first precise definitions of terms such as Friendly AI and Seed AI.
  • "Levels of Organization in General Intelligence" (2002): Analyzes AGI through its decomposition in five subsystems, successive levels of functional organization: Code, sensory modalities, concepts, thoughts, and deliberation. Also discusses some advantages artificial minds would have, such as the possibility of Recursive self-improvement.
  • "Coherent Extrapolated Volition"(2004): Presents the difficulties and possible solutions for incorporating friendliness into an AGI. It argues that making an AGI simply do what we tell it to could be dangerous, since we don't know what we want. Instead we should program the AGI to do what we want, predicting what the vectorial sum of an idealized version of what humanity would want, "if we knew more, thought faster, were more the people we wished we were, had grown up farther together”. He calls this the coherent extrapolated volition of humankind, or CEV.
  • "Timeless Decision Theory" (2010): Describes Timeless decision theory, ”an extension of causal decision networks that compactly represents uncertainty about correlated computational processes and represents the decision maker as such a process”. It solves many problems which Causal Decision Theory or Evidential Decision Theory don't have a plausible solution: Newcomb's problem, Solomon's Problems and Prisoner's dilemma.
  • "Complex Value Systems are Required to Realize Valuable Futures" (2011): Discusses the Complexity of value: we can’t come up with a simple rule or description that sums up all human values. It analyzes how this problem makes it difficult to build a valuable future.

Links