Difference between revisions of "Coherent Extrapolated Volition"

From Lesswrongwiki
Jump to: navigation, search
m (References)
m
Line 1: Line 1:
On the topic of [[Friendly AI]], [[Eliezer Yudkowsky]] argues that it would not be sufficient to explicitly program our desires into an AI. Rather, we must  To truly act in our best interests, a superintelligent AI must do what we ''want'' it to do, rather than what we ''tell'' it to do. Therefore, what we need is a meta-level method of determining what the object-level goal will be. Yudkowsky has presented [http://intelligence.org/upload/CEV.html Coherent Extrapolated Volition] as an example of what the solution to this problem would look like.
+
On the topic of [[Friendly AI]], [[Eliezer Yudkowsky]] argues that it would not be sufficient to explicitly program our desires into an AI. Rather, we must  To truly act in our best interests, a superintelligent AI must do what we ''want'' it to do, rather than what we ''tell'' it to do. Therefore, what we need is a meta-level method of determining what the object-level goal will be. Yudkowsky has presented [http://intelligence.org/files/CEV.pdf Coherent Extrapolated Volition] as an example of what the solution to this problem would look like.
  
 
In calculating CEV, an AI would predict what an idealized version of us would want, "if we knew more, thought faster, were more the people we wished we were, had grown up farther together". It would recursively iterate this prediction for humanity as a whole, and determine the desires which converge. This initial dynamic would be used to form the AI's utility function.
 
In calculating CEV, an AI would predict what an idealized version of us would want, "if we knew more, thought faster, were more the people we wished we were, had grown up farther together". It would recursively iterate this prediction for humanity as a whole, and determine the desires which converge. This initial dynamic would be used to form the AI's utility function.
Line 6: Line 6:
  
 
==Blog posts==
 
==Blog posts==
 +
* [http://www.acceleratingfuture.com/michael/blog/2009/12/a-short-introduction-to-coherent-extrapolated-volition-cev/ A Short Introduction to Coherent Extrapolated Volition] by Michael Anissimov
 
* [http://lesswrong.com/lw/2b7/hacking_the_cev_for_fun_and_profit/ Hacking the CEV for Fun and Profit] by Wei Dai
 
* [http://lesswrong.com/lw/2b7/hacking_the_cev_for_fun_and_profit/ Hacking the CEV for Fun and Profit] by Wei Dai
 
* [http://lesswrong.com/lw/3fn/two_questions_about_cev_that_worry_me/ Two questions about CEV that worry me] by Vladimir Slepnev
 
* [http://lesswrong.com/lw/3fn/two_questions_about_cev_that_worry_me/ Two questions about CEV that worry me] by Vladimir Slepnev

Revision as of 02:16, 14 July 2012

On the topic of Friendly AI, Eliezer Yudkowsky argues that it would not be sufficient to explicitly program our desires into an AI. Rather, we must To truly act in our best interests, a superintelligent AI must do what we want it to do, rather than what we tell it to do. Therefore, what we need is a meta-level method of determining what the object-level goal will be. Yudkowsky has presented Coherent Extrapolated Volition as an example of what the solution to this problem would look like.

In calculating CEV, an AI would predict what an idealized version of us would want, "if we knew more, thought faster, were more the people we wished we were, had grown up farther together". It would recursively iterate this prediction for humanity as a whole, and determine the desires which converge. This initial dynamic would be used to form the AI's utility function.

Problems with CEV include the great difficulty of implementing such a program, and the potential that human values may not converge. Yudkowsky considered CEV obsolete almost immediately after its publication in 2004. Despite this, criticism of CEV has been a popular topic of criticism on LessWrong.

Blog posts

See also

References