Difference between revisions of "Complexity of value"

From Lesswrongwiki
Jump to: navigation, search
m (Flow)
Line 1: Line 1:
The thesis that human values have high [[Kolmogorov complexity]] - our [[preference]]s, the things we care about, ''don't'' compress down to one simple rule, or a few simple rules.
+
Complexity of value is the thesis that human values have high [[Kolmogorov complexity]]; that our [[preferences]], the things we care about, cannot be summed by a few simple rules, or compressed.  
  
In general, human choices do compress; someone who wants to survive will, over the course of a lifetime, take many different actions, and pursue many different goals, in order to surviveIn this sense, human choices definitely compress beyond the raw list of actions.
+
Many human choices can be compressed, by representing them by simple rules - the desire to survive produces innumerable actions and subgoals as we fulfil that desire.  But people don't ''just'' want to survive - although you can compress many human activities to that desire, you cannot compress all of human existence into it.  The human equivalents of a utility function, our terminal values, contain many different elements that are not strictly reducible to one another.  William Frankena offered [http://plato.stanford.edu/entries/value-intrinsic-extrinsic/#WhaHasIntVal this list] of things which many cultures and people seem to value (for their own sake rather than strictly for their external consequences):
 
 
But people don't ''just'' want to survive - although you can compress many human activities to that desire, you cannot compress all of human existence into it.  The human equivalents of a utility function, our terminal values, contain many different elements that are not strictly reducible to one another.  William Frankena offered [http://plato.stanford.edu/entries/value-intrinsic-extrinsic/#WhaHasIntVal this list] of things which many cultures and people seem to value (for their own sake rather than strictly for their external consequences):
 
  
 
:"Life, consciousness, and activity; health and strength; pleasures and satisfactions of all or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true opinions of various kinds, understanding, wisdom; beauty, harmony, proportion in objects contemplated; aesthetic experience; morally good dispositions or virtues; mutual affection, love, friendship, cooperation; just distribution of goods and evils; harmony and proportion in one's own life; power and experiences of achievement; self-expression; freedom; peace, security; adventure and novelty; and good reputation, honor, esteem, etc."
 
:"Life, consciousness, and activity; health and strength; pleasures and satisfactions of all or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true opinions of various kinds, understanding, wisdom; beauty, harmony, proportion in objects contemplated; aesthetic experience; morally good dispositions or virtues; mutual affection, love, friendship, cooperation; just distribution of goods and evils; harmony and proportion in one's own life; power and experiences of achievement; self-expression; freedom; peace, security; adventure and novelty; and good reputation, honor, esteem, etc."

Revision as of 13:00, 14 July 2012

Complexity of value is the thesis that human values have high Kolmogorov complexity; that our preferences, the things we care about, cannot be summed by a few simple rules, or compressed.

Many human choices can be compressed, by representing them by simple rules - the desire to survive produces innumerable actions and subgoals as we fulfil that desire. But people don't just want to survive - although you can compress many human activities to that desire, you cannot compress all of human existence into it. The human equivalents of a utility function, our terminal values, contain many different elements that are not strictly reducible to one another. William Frankena offered this list of things which many cultures and people seem to value (for their own sake rather than strictly for their external consequences):

"Life, consciousness, and activity; health and strength; pleasures and satisfactions of all or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true opinions of various kinds, understanding, wisdom; beauty, harmony, proportion in objects contemplated; aesthetic experience; morally good dispositions or virtues; mutual affection, love, friendship, cooperation; just distribution of goods and evils; harmony and proportion in one's own life; power and experiences of achievement; self-expression; freedom; peace, security; adventure and novelty; and good reputation, honor, esteem, etc."

The complexity of value is a major theme of Eliezer Yudkowsky's writing for two reasons:

  • Caricatures of rationalists often have them moved by artificially simplified values - for example, only caring about personal pleasure. This becomes a template for arguing against rationality: X is valuable, but rationality says to only care about Y, in which case we could not value X, therefore do not be rational.
  • Underestimating the complexity of value leads to underestimating the difficulty of Friendly AI; and there are notable cognitive biases and fallacies which lead people to underestimate this complexity.

Major posts

Other posts

See also