Difference between revisions of "Value extrapolation"

From Lesswrongwiki
Jump to: navigation, search
(Bit on a couple suggestions)
Line 1: Line 1:
 
'''Value extrapolation''' is the process of determining the true values of a person. This task is quite challenging, due to the [[Complexity of value| complexity of human values]].
 
'''Value extrapolation''' is the process of determining the true values of a person. This task is quite challenging, due to the [[Complexity of value| complexity of human values]].
  
[[Coherent Extrapolated Volition]] is one proposed method of determining the values of humanity as a whole for a [[friendly AI]] to respect. Others have argued that the [[biases]] of the human mind are great that there is no meaningful difference between a human value and a human error.  
+
[[Coherent Extrapolated Volition]] is one proposed method of determining the values of humanity as a whole for a [[friendly AI]] to respect. [[Coherent Aggregated Volition]] is a proposal by [[Ben Goertzel]] similar to CEV but which aggregates and averages rather than extrapolates the values that humans have today. It is intended to be simpler and less controversial than CEV but would solve no inconsistencies in our values and instead cement them permanently.
 +
 
 +
Phil Goetz has argued that the [[biases]] of the human mind are so great that there is no meaningful difference between a human value and a human error, though he believes this can be partially avoided by accepting an arbitrary ethical base. Mitchell Porter optimistically argues that as [[AGI skepticism]] fades the field of [[machine ethics]] will experience huge growth, and that the public will engage in the problem and develop an aggregate set of values to program an [[AGI]] with.  
  
 
Paul Christiano has suggested that a set of extrapolated values may be created using [[WBE]], and running emulated people at high speeds until they settles on a set of values. These people, not having to worry about existential threats, would make better decisions then us. He argues the threat of [[existential risks]] merits using less than perfect value extrapolation. This has been criticized as simply passing the problem on, however.  
 
Paul Christiano has suggested that a set of extrapolated values may be created using [[WBE]], and running emulated people at high speeds until they settles on a set of values. These people, not having to worry about existential threats, would make better decisions then us. He argues the threat of [[existential risks]] merits using less than perfect value extrapolation. This has been criticized as simply passing the problem on, however.  
Line 13: Line 15:
 
== Blog posts ==
 
== Blog posts ==
 
*[http://ordinaryideas.wordpress.com/2012/04/21/indirect-normativity-write-up/ “Indirect Normativity” Write-up] by paulfchristiano, [http://lesswrong.com/lw/c0k/formalizing_value_extrapolation/ LW post and comments]  
 
*[http://ordinaryideas.wordpress.com/2012/04/21/indirect-normativity-write-up/ “Indirect Normativity” Write-up] by paulfchristiano, [http://lesswrong.com/lw/c0k/formalizing_value_extrapolation/ LW post and comments]  
 +
*[http://multiverseaccordingtoben.blogspot.ca/2010/03/coherent-aggregated-volition-toward.html Coherent Aggregated Volition: A Method for Deriving Goal System Content for Advanced, Beneficial AGIs] by Ben Goertzel
 
*[http://lesswrong.com/lw/c1x/extrapolating_values_without_outsourcing/ Extrapolating values without outsourcing]
 
*[http://lesswrong.com/lw/c1x/extrapolating_values_without_outsourcing/ Extrapolating values without outsourcing]
 
*[http://lesswrong.com/lw/y3/value_is_fragile/ Value is Fragile]
 
*[http://lesswrong.com/lw/y3/value_is_fragile/ Value is Fragile]

Revision as of 12:59, 21 July 2012

Value extrapolation is the process of determining the true values of a person. This task is quite challenging, due to the complexity of human values.

Coherent Extrapolated Volition is one proposed method of determining the values of humanity as a whole for a friendly AI to respect. Coherent Aggregated Volition is a proposal by Ben Goertzel similar to CEV but which aggregates and averages rather than extrapolates the values that humans have today. It is intended to be simpler and less controversial than CEV but would solve no inconsistencies in our values and instead cement them permanently.

Phil Goetz has argued that the biases of the human mind are so great that there is no meaningful difference between a human value and a human error, though he believes this can be partially avoided by accepting an arbitrary ethical base. Mitchell Porter optimistically argues that as AGI skepticism fades the field of machine ethics will experience huge growth, and that the public will engage in the problem and develop an aggregate set of values to program an AGI with.

Paul Christiano has suggested that a set of extrapolated values may be created using WBE, and running emulated people at high speeds until they settles on a set of values. These people, not having to worry about existential threats, would make better decisions then us. He argues the threat of existential risks merits using less than perfect value extrapolation. This has been criticized as simply passing the problem on, however.

See Also

Blog posts

External Links