Difference between revisions of "Fragility of value"

From Lesswrongwiki
Jump to: navigation, search
(First draft)
(Redirected page to Complexity of value)
(12 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Human values are fickle things. They are the product of our evolutionary history, and more recently predominantly our social heritage. We have not yet compared values with any other intelligent life, and most of us find those values in our history repulsive. With no inherent morality (see the [[metaethics sequence]], in particular [http://lesswrong.com/lw/sb/could_anything_be_right/ these] [http://lesswrong.com/lw/sm/the_meaning_of_right/ three] [http://lesswrong.com/lw/rr/the_moral_void/ posts]), a person's values define them. Whatever else an [[AGI]] does, it must not change our values - [[the utility function is not up for grabs]].
#REDIRECT [[Complexity of value]]
A poorly constrained [[artificial general intelligence]] seeking to maximize the [[utility]] experienced by humanity might turn us into blobs of perpetual orgasm. A world with immortality designed without [[fun theory]] might find its citizens modifying themselves as to find utterly engrossing a pursuit such as making table legs.
== See Also ==
* [[Metaethics sequence]]
*[[Fun theory]]
*[[The utility function is not up for grabs]]
== Blog Posts ==
*[http://lesswrong.com/lw/y3/value_is_fragile/ Value is Fragile]

Latest revision as of 07:20, 3 October 2012