Difference between revisions of "Coherent Aggregated Volition"

From Lesswrongwiki
Jump to: navigation, search
Line 1: Line 1:
'''Coherent Aggregated Volition''' is [[Ben Goertzel]]'s response to [[Eliezer Yudkowsky]]'s [[Coherent Extrapolated Volition]]. CAV would be a combination of the goals and beliefs of humanity at the present time.  
+
'''Coherent Aggregated Volition''' is one of [[Ben Goertzel]]'s responses to [[Eliezer Yudkowsky]]'s [[Coherent Extrapolated Volition]], the other being [[Coherent Blended Volition]]. CAV would be a combination of the goals and beliefs of humanity at the present time.  
  
The author considers the "extrapolation" aspect of CEV as distorting the concept of volition and to be highly uncertain. He considers that if the person whose volition is being extrapolated has some inconsistent aspects (which is tipically human), then there could be a great variety of possible extrapolations. The problem would then be which version of this extrapolated human to choose, or how to aggregate them, which would be very difficult to achieve.
+
The author considers the "extrapolation" aspect of CEV as distorting the concept of volition and to be highly uncertain. He considers that if the person whose volition is being extrapolated has some inconsistent aspects (which is typically human), then there could be a great variety of possible extrapolations. The problem would then be which version of this extrapolated human to choose, or how to aggregate them, which would be very difficult to achieve.
  
Coherent Aggregated Volition is presented as simpler than his interpretation of CEV, and intended to be easier to formalize and prototype in the foreseeable future (with the help of platforms such as [http://opencog.org OpenCog]. CAV is not, however, intended to answer the question of Friendly AI, although Goertzel claims CEV is possibly not the answer as well.
+
Coherent Aggregated Volition is presented as simpler than his interpretation of CEV, and intended to be easier to formalize and prototype in the foreseeable future (with the help of platforms such as [http://opencog.org OpenCog]. CAV is not, however, intended to answer the question of provably [[Friendly AI]], although Goertzel claims CEV may not answer question either.
  
 
==The concept==
 
==The concept==
The author starts by defending that we must treat goals and beliefs together, as a single concept, which he calls ''gobs'' (and ''gobses'' for the plural). Each agent thus can have several ''gobs'', logically consistent or not. As a way to measure how much these ''gobs'' are distant to each other, the term ''gobs metric'' is used - the persons or AGI could agree, with various degrees, on several metrics, but it seems probable that this individual's metrics would differ less than their ''gobses''.
+
The author starts by arguing that we must treat goals and beliefs together, as a single concept, which he calls ''gobs'' (and ''gobses'' for the plural). Each agent thus can have several ''gobs'', logically consistent or not. As a way of measuring the distance of these ''gobs'' from each other, the term ''gobs metric'' is used - the persons or AGI could agree, with various degrees, on several metrics, but it seems probable that this individual's metrics would differ less than their ''gobses''.
  
 
Then, given a population of intelligent agents with different ''gobses'', we could then try to find a single ''gobs'' that maximizes logical consistency, compactness, similarity to the different ''gobses'' in the population and amount of evidence supporting these beliefs. This "multi-extremal optimization algorithm" is what he calls Coherent Aggregated Volition. The term expresses the attempt to achieve both coherence and an aggregation of the population volitions.
 
Then, given a population of intelligent agents with different ''gobses'', we could then try to find a single ''gobs'' that maximizes logical consistency, compactness, similarity to the different ''gobses'' in the population and amount of evidence supporting these beliefs. This "multi-extremal optimization algorithm" is what he calls Coherent Aggregated Volition. The term expresses the attempt to achieve both coherence and an aggregation of the population volitions.
Line 15: Line 15:
 
Although CEV is seen as possibly giving a feasible solution, Goertzel states there's no guarantee of this, and that Yudkowsky's method can generate solutions very far from the population's ''gobses''.
 
Although CEV is seen as possibly giving a feasible solution, Goertzel states there's no guarantee of this, and that Yudkowsky's method can generate solutions very far from the population's ''gobses''.
  
In some experiments with iteratively repairs within a probabilistic reasoning system, the author claims it seems we can reach a set of beliefs very different from the one we started - which seems to reflect the CEV process and main problem. That is, the iterative refinement of the agents' goals and beliefs can not always be a good system to turn inconsistent values with similar consistent ones.
+
In some experiments with iteratively repairing inconsistent beliefs within a probabilistic reasoning system, the author claims that it seems like we can reach a set of beliefs very different from the one we started - which may be a problem with CEV. That is, the iterative refinement of the agents' goals and beliefs might not always be a good way to turn inconsistent values into similar consistent ones.
  
Finally, while it seems that CEV bypasses the ''humanity'' feeling, by not encompassing the person's growth process and inconsistencies in the human mind, CAV tries to summarize it, respecting and not replacing it. The intrinsic inheritance of possibly more "bad" aspects of humanity than CEV is then seen as more ''human''.
+
Finally, Goertzel feels that it seems like CEV bypasses an essential aspect of being human, by not allowing humans to resolve their inconsistencies themselves. CAV tries to summarize this process, respecting and not replacing it, even if this leads to more "bad" aspects of humanity being retained.
  
 
==See also==
 
==See also==

Revision as of 16:48, 3 October 2012

Coherent Aggregated Volition is one of Ben Goertzel's responses to Eliezer Yudkowsky's Coherent Extrapolated Volition, the other being Coherent Blended Volition. CAV would be a combination of the goals and beliefs of humanity at the present time.

The author considers the "extrapolation" aspect of CEV as distorting the concept of volition and to be highly uncertain. He considers that if the person whose volition is being extrapolated has some inconsistent aspects (which is typically human), then there could be a great variety of possible extrapolations. The problem would then be which version of this extrapolated human to choose, or how to aggregate them, which would be very difficult to achieve.

Coherent Aggregated Volition is presented as simpler than his interpretation of CEV, and intended to be easier to formalize and prototype in the foreseeable future (with the help of platforms such as OpenCog. CAV is not, however, intended to answer the question of provably Friendly AI, although Goertzel claims CEV may not answer question either.

The concept

The author starts by arguing that we must treat goals and beliefs together, as a single concept, which he calls gobs (and gobses for the plural). Each agent thus can have several gobs, logically consistent or not. As a way of measuring the distance of these gobs from each other, the term gobs metric is used - the persons or AGI could agree, with various degrees, on several metrics, but it seems probable that this individual's metrics would differ less than their gobses.

Then, given a population of intelligent agents with different gobses, we could then try to find a single gobs that maximizes logical consistency, compactness, similarity to the different gobses in the population and amount of evidence supporting these beliefs. This "multi-extremal optimization algorithm" is what he calls Coherent Aggregated Volition. The term expresses the attempt to achieve both coherence and an aggregation of the population volitions.

CAV has some free parameters, like the averaging method, the measure of compactness, consistency evaluation and so on, but these are seen as features rather than limitations and do not taint the simplicity of the idea. At the same time, it is possible to refine some of the criteria stated before without changing the nature of the method.

CEV vs CAV

Although CEV is seen as possibly giving a feasible solution, Goertzel states there's no guarantee of this, and that Yudkowsky's method can generate solutions very far from the population's gobses.

In some experiments with iteratively repairing inconsistent beliefs within a probabilistic reasoning system, the author claims that it seems like we can reach a set of beliefs very different from the one we started - which may be a problem with CEV. That is, the iterative refinement of the agents' goals and beliefs might not always be a good way to turn inconsistent values into similar consistent ones.

Finally, Goertzel feels that it seems like CEV bypasses an essential aspect of being human, by not allowing humans to resolve their inconsistencies themselves. CAV tries to summarize this process, respecting and not replacing it, even if this leads to more "bad" aspects of humanity being retained.

See also

References