Difference between revisions of "The Hanson-Yudkowsky AI-Foom Debate"

From Lesswrongwiki
Jump to: navigation, search
m (Prologue: bracket fix)
Line 5: Line 5:
 
==Prologue==
 
==Prologue==
  
* [http://www.overcomingbias.com/2008/11/fund-ubertool.html Fund UberTool?] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/fund-ubertool.html Fund UberTool?] by [[Robin Hanson]]
* [http://www.overcomingbias.com/2008/11/engelbarts-uber.html Engelbart as UberTool?] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/engelbarts-uber.html Engelbart as UberTool?] by [[Robin Hanson]]
* [http://www.overcomingbias.com/2008/11/englebart-not-r.html Friendly Teams] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/englebart-not-r.html Friendly Teams] by [[Robin Hanson]]
* [http://www.overcomingbias.com/2008/11/friendliness-fa.html Friendliness Factors] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/friendliness-fa.html Friendliness Factors] by [[Robin Hanson]]
* [http://lesswrong.com/lw/vz/the_weak_inside_view/ The Weak Inside View] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/vz/the_weak_inside_view/ The Weak Inside View] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/11/setting-the-sta.html Setting The Stage] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/setting-the-sta.html Setting The Stage] by [[Robin Hanson]]
* [http://lesswrong.com/lw/w0/the_first_world_takeover/ The First World Takeover] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/w0/the_first_world_takeover/ The First World Takeover] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/11/abstraction-vs.html Abstraction, Not Analogy] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/abstraction-vs.html Abstraction, Not Analogy] by [[Robin Hanson]]
* [http://lesswrong.com/lw/w1/whence_your_abstractions/ Whence Your Abstractions?] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/w1/whence_your_abstractions/ Whence Your Abstractions?] by [[Eliezer Yudkowsky]]
  
 
==Main Sequence==
 
==Main Sequence==
  
* [http://www.overcomingbias.com/2008/11/ai-go-foom.html AI Go Foom] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/ai-go-foom.html AI Go Foom] by [[Robin Hanson]]
* [http://lesswrong.com/lw/w2/observing_optimization/ Observing Optimization] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/w2/observing_optimization/ Observing Optimization] by [[Eliezer Yudkowsky]]
* [http://lesswrong.com/lw/w3/lifes_story_continues/ Life’s Story Continues] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/w3/lifes_story_continues/ Life’s Story Continues] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/11/emulations-go-f.html Emulations Go Foom] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/emulations-go-f.html Emulations Go Foom] by [[Robin Hanson]]
 
* [http://www.overcomingbias.com/2008/11/brain-emulation.html Brain Emulation and Hard Takeoff] by [[Carl Shulman]]
 
* [http://www.overcomingbias.com/2008/11/brain-emulation.html Brain Emulation and Hard Takeoff] by [[Carl Shulman]]
 
* [http://www.overcomingbias.com/2008/11/billion-dollar.html Billion Dollar Bots] by [[James Miller]]
 
* [http://www.overcomingbias.com/2008/11/billion-dollar.html Billion Dollar Bots] by [[James Miller]]
* [http://lesswrong.com/lw/w4/surprised_by_brains/ Surprised by Brains] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/w4/surprised_by_brains/ Surprised by Brains] by [[Eliezer Yudkowsky]]
 
* [http://www.overcomingbias.com/2008/11/suppose-that-ro.html “Evicting” brain emulations] by [[Carl Shulman]]
 
* [http://www.overcomingbias.com/2008/11/suppose-that-ro.html “Evicting” brain emulations] by [[Carl Shulman]]
* [http://lesswrong.com/lw/w5/cascades_cycles_insight/ Cascades, Cycles, Insight…] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/w5/cascades_cycles_insight/ Cascades, Cycles, Insight…] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/11/when-life-is-ch.html When Life Is Cheap, Death Is Cheap] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/when-life-is-ch.html When Life Is Cheap, Death Is Cheap] by [[Robin Hanson]]
* [http://lesswrong.com/lw/w6/recursion_magic/ …Recursion, Magic] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/w6/recursion_magic/ …Recursion, Magic] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/11/abstractdistant.html Abstract/Distant Future Bias] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/abstractdistant.html Abstract/Distant Future Bias] by [[Robin Hanson]]
* [http://lesswrong.com/lw/w8/engelbart_insufficiently_recursive/ Engelbart: Insufficiently Recursive] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/w8/engelbart_insufficiently_recursive/ Engelbart: Insufficiently Recursive] by [[Eliezer Yudkowsky]]
* [http://lesswrong.com/lw/w9/total_nano_domination/ Total Nano Domination] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/w9/total_nano_domination/ Total Nano Domination] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/11/dreams-of-autar.html Dreams of Autarky] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/dreams-of-autar.html Dreams of Autarky] by [[Robin Hanson]]
* [http://www.overcomingbias.com/2008/11/total-tech-wars.html Total Tech Wars] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/total-tech-wars.html Total Tech Wars] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wc/singletons_rule_ok/ Singletons Rule OK] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wc/singletons_rule_ok/ Singletons Rule OK] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/11/stuck-in-throat.html Stuck In Throat] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/11/stuck-in-throat.html Stuck In Throat] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wd/disappointment_in_the_future/ Disappointment in the Future] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wd/disappointment_in_the_future/ Disappointment in the Future] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/12/i-heart-cyc.html I Heart CYC] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/i-heart-cyc.html I Heart CYC] by [[Robin Hanson]]
* [http://lesswrong.com/lw/we/recursive_selfimprovement/ Recursive Self-Improvement] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/we/recursive_selfimprovement/ Recursive Self-Improvement] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/12/whither-manufac.html Whither Manufacturing?] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/whither-manufac.html Whither Manufacturing?] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wf/hard_takeoff/ Hard Takeoff] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wf/hard_takeoff/ Hard Takeoff] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/12/test-near-apply.html Test Near, Apply Far] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/test-near-apply.html Test Near, Apply Far] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wg/permitted_possibilities_locality/ Permitted Possibilities, & Locality] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wg/permitted_possibilities_locality/ Permitted Possibilities, & Locality] by [[Eliezer Yudkowsky]]
* [http://lesswrong.com/lw/wh/underconstrained_abstractions/ Underconstrained Abstractions] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wh/underconstrained_abstractions/ Underconstrained Abstractions] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/12/beware-hockey-s.html Beware Hockey Stick Plans] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/beware-hockey-s.html Beware Hockey Stick Plans] by [[Robin Hanson]]
* [http://www.overcomingbias.com/2008/12/evolved-desires.html Evolved Desires] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/evolved-desires.html Evolved Desires] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wi/sustained_strong_recursion/ Sustained Strong Recursion] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wi/sustained_strong_recursion/ Sustained Strong Recursion] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/12/friendly-projec.html Friendly Projects vs. Products] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/friendly-projec.html Friendly Projects vs. Products] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wj/is_that_your_true_rejection/ Is That Your True Rejection?] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wj/is_that_your_true_rejection/ Is That Your True Rejection?] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/12/shared-ai-wins.html Shared AI Wins] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/shared-ai-wins.html Shared AI Wins] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wk/artificial_mysterious_intelligence/ Artificial Mysterious Intelligence] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wk/artificial_mysterious_intelligence/ Artificial Mysterious Intelligence] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/12/wrapping-up.html Wrapping Up] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/wrapping-up.html Wrapping Up] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wl/true_sources_of_disagreement/ True Sources of Disagreement] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wl/true_sources_of_disagreement/ True Sources of Disagreement] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html The Bad Guy Bias] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html The Bad Guy Bias] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wm/disjunctions_antipredictions_etc/ Disjunctions, Antipredictions, Etc.] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wm/disjunctions_antipredictions_etc/ Disjunctions, Antipredictions, Etc.] by [[Eliezer Yudkowsky]]
* [http://www.overcomingbias.com/2008/12/are-ais-homo-ec.html Are AIs Homo Economicus?] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/are-ais-homo-ec.html Are AIs Homo Economicus?] by [[Robin Hanson]]
* [http://www.overcomingbias.com/2008/12/two-visions-of.html Two Visions Of Heritage] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/two-visions-of.html Two Visions Of Heritage] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wo/the_mechanics_of_disagreement/ The Mechanics of Disagreement] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wo/the_mechanics_of_disagreement/ The Mechanics of Disagreement] by [[Eliezer Yudkowsky]]
  
 
==Conclusion==
 
==Conclusion==
  
* [http://www.overcomingbias.com/2008/12/what-core-argument.html What Core Argument?] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/what-core-argument.html What Core Argument?] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wp/what_i_think_if_not_why/ What I Think, If Not Why] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wp/what_i_think_if_not_why/ What I Think, If Not Why] by [[Eliezer Yudkowsky]]
  
 
==Postscript==
 
==Postscript==
  
* [http://www.overcomingbias.com/2008/12/we-agree-get-froze.html We Agree: Get Froze] by Robin Hanson
+
* [http://www.overcomingbias.com/2008/12/we-agree-get-froze.html We Agree: Get Froze] by [[Robin Hanson]]
* [http://lesswrong.com/lw/wq/you_only_live_twice/ You Only Live Twice] by Eliezer Yudkowsky
+
* [http://lesswrong.com/lw/wq/you_only_live_twice/ You Only Live Twice] by [[Eliezer Yudkowsky]]
  
 
[[Category:Sequences]]
 
[[Category:Sequences]]

Revision as of 08:36, 5 January 2010

In late 2008, an extensive and long-awaited debate about the Technological Singularity occurred on Overcoming Bias, mainly between Robin Hanson and Eliezer Yudkowsky. It focused on the likelihood of hard AI takeoff, the need for a theory of Friendliness, and the future of AI, brain emulations, and recursive improvement in general. This debate is often used to illustrate the difficulty of resolving disagreements, even among expert rationalists.

The posts constituting this debate have been collected here to make it easier to follow for future audiences.

Prologue

Main Sequence

Conclusion

Postscript