The Hanson-Yudkowsky AI-Foom Debate
In late 2008, an extensive and long-awaited debate about the Technological Singularity occurred on Overcoming Bias, mainly between Robin Hanson and Eliezer Yudkowsky. It focused on the likelihood of hard AI takeoff ("FOOM"), the need for a theory of Friendliness, and the future of AI, brain emulations, and recursive improvement in general. A stand-up debate on the same themes took place at Jane Street Capital on Wednesday June 30 2011. Kaj Sotala made available an incomplete draft summary and analysis of the debate in November 2012.
This debate is often used to illustrate the difficulty of resolving disagreements, even among expert rationalists.
The posts constituting this debate have been collected here to make it easier to follow for future audiences, alongside links to the Jane Street debate.
Prologue
- Fund UberTool? by Robin Hanson
- Engelbart as UberTool? by Robin Hanson
- Friendly Teams by Robin Hanson
- Friendliness Factors by Robin Hanson
- The Weak Inside View by Eliezer Yudkowsky
- Setting The Stage by Robin Hanson
- The First World Takeover by Eliezer Yudkowsky
- Abstraction, Not Analogy by Robin Hanson
- Whence Your Abstractions? by Eliezer Yudkowsky
Main sequence
- AI Go Foom by Robin Hanson
- Optimization and the Singularity by Eliezer Yudkowsky
- Eliezer's Meta-Level Determinismi by Robin Hanson
- Observing Optimization by Eliezer Yudkowsky
- Life's Story Continues by Eliezer Yudkowsky
- Emulations Go Foom by Robin Hanson
- Brain Emulation and Hard Takeoff by Carl Shulman
- Billion Dollar Bots by James Miller
- Surprised by Brains by Eliezer Yudkowsky
- “Evicting” brain emulations by Carl Shulman
- Cascades, Cycles, Insight... by Eliezer Yudkowsky
- When Life Is Cheap, Death Is Cheap by Robin Hanson
- ...Recursion, Magic by Eliezer Yudkowsky
- Abstract/Distant Future Bias by Robin Hanson
- Engelbart: Insufficiently Recursive by Eliezer Yudkowsky
- Total Nano Domination by Eliezer Yudkowsky
- Dreams of Autarky by Robin Hanson
- Total Tech Wars by Robin Hanson
- Singletons Rule OK by Eliezer Yudkowsky
- Stuck In Throat by Robin Hanson
- Disappointment in the Future by Eliezer Yudkowsky
- I Heart CYC by Robin Hanson
- Recursive Self-Improvement by Eliezer Yudkowsky
- Whither Manufacturing? by Robin Hanson
- Hard Takeoff by Eliezer Yudkowsky
- Test Near, Apply Far by Robin Hanson
- Permitted Possibilities, & Locality by Eliezer Yudkowsky
- Underconstrained Abstractions by Eliezer Yudkowsky
- Beware Hockey Stick Plans by Robin Hanson
- Evolved Desires by Robin Hanson
- Sustained Strong Recursion by Eliezer Yudkowsky
- Friendly Projects vs. Products by Robin Hanson
- Is That Your True Rejection? by Eliezer Yudkowsky
- Shared AI Wins by Robin Hanson
- Artificial Mysterious Intelligence by Eliezer Yudkowsky
- Wrapping Up by Robin Hanson
- True Sources of Disagreement by Eliezer Yudkowsky
- The Bad Guy Bias by Robin Hanson
- Disjunctions, Antipredictions, Etc. by Eliezer Yudkowsky
- Are AIs Homo Economicus? by Robin Hanson
- Two Visions Of Heritage by Robin Hanson
- The Mechanics of Disagreement by Eliezer Yudkowsky
Conclusion
- What Core Argument? by Robin Hanson
- What I Think, If Not Why by Eliezer Yudkowsky
- Not Taking Over the World by Eliezer Yudkowsky
Postscript
Jane Street debate
- Debating Yudkowsky by Robin Hanson
- Video with better quality audio 1.57G
- Poor quality WMA audio 97M
- Poor quality MP3 audio 97M
- Partial transcript by Chris Hallquist
