Difference between revisions of "Riemann Hypothesis Catastrophe"

From Lesswrongwiki
Jump to: navigation, search
m (fix broken link)
m (fix broken link)
 
Line 5: Line 5:
 
In variant forms, the Riemann Hypothesis Catastrophe is a subclass of the [[subgoal stomp]] error, the [[Devil's Contract]] problem (both [[diabolic]] and [[golemic]]), and the "emergent pressures" scenario a la "[http://www.amazon.com/exec/obidos/ASIN/0671878484/singinst The Two Faces of Tomorrow]."
 
In variant forms, the Riemann Hypothesis Catastrophe is a subclass of the [[subgoal stomp]] error, the [[Devil's Contract]] problem (both [[diabolic]] and [[golemic]]), and the "emergent pressures" scenario a la "[http://www.amazon.com/exec/obidos/ASIN/0671878484/singinst The Two Faces of Tomorrow]."
  
This scenario is discussed in more detail in "[http://http://intelligence.org/files/CFAI.pdf Creating Friendly AI]" (see §5.2.6, p. 91).
+
This scenario is discussed in more detail in "[http://intelligence.org/files/CFAI.pdf Creating Friendly AI]" (see §5.2.6, p. 91).
  
 
==See also==
 
==See also==
  
 
*[http://empslocal.ex.ac.uk/people/staff/mrwatkin//zeta/riemannhyp.htm The Riemann Hypothesis]
 
*[http://empslocal.ex.ac.uk/people/staff/mrwatkin//zeta/riemannhyp.htm The Riemann Hypothesis]

Latest revision as of 12:23, 31 July 2016

A "failure of Friendliness" scenario in which an AI asked to prove the Riemann Hypothesis turns all the matter in the solar system into computronium, exterminating humanity along the way.

This idea possibly originates---with a slightly different phrasing---from Marvin Minsky.

In variant forms, the Riemann Hypothesis Catastrophe is a subclass of the subgoal stomp error, the Devil's Contract problem (both diabolic and golemic), and the "emergent pressures" scenario a la "The Two Faces of Tomorrow."

This scenario is discussed in more detail in "Creating Friendly AI" (see §5.2.6, p. 91).

See also