Value extrapolation is the process of determining the true values of a person. This task is quite challenging, due to the complexity of human values.
Coherent Extrapolated Volition is one proposed method of determining the values of humanity as a whole for a friendly AI to respect. Others have argued that the biases of the human mind are great that there is no meaningful difference between a human value and a human error.
Paul Christiano has suggested that a set of extrapolated values may be created using WBE, and running emulated people at high speeds until they settles on a set of values. These people, not having to worry about existential threats, would make better decisions then us. He argues the threat of existential risks merits using less than perfect value extrapolation. This has been criticized as simply passing the problem on, however.
- “Indirect Normativity” Write-up by paulfchristiano, LW post and comments
- Extrapolating values without outsourcing
- Value is Fragile
- Human errors, human values