Fun theory addresses this problem: If we can arrange to live for a very long time with every increasing intelligence, how can we continue to have any fun. When we have intellects that can deeply comprehend every movie, novel, and concert ever created in an instant, that intimately know every twist of every forest path, and with enhanced bodies can also swim under the Atlantic in a day and navigate a snowboard down Mount Everest as easily as you or I could steer a bicycle down the street, what will remain?
Unless we can answer this question, we might be faced by endless boredom. This could be seen as an argument against key hopes of transhumanism, such as lifespan extension, human intelligence enhancement, and physical enhancement.
Transhumanists work towards a better future, but Utopians in the past have generally imagined futures that no one would want to live in. George Orwell commented on the inability of such authors to imagine futures where anyone would actually *want* to live. It is a commonplace that the Christian Heaven, as usually portrayed, would attract nobody. Almost all Christian writers dealing with Heaven either say frankly that it is indescribable or conjure up a vague picture of gold, precious stones, and the endless singing of hymns. This has, it is true, inspired some of the best poems in the world: Thy walls are of chalcedony, Thy bulwarks diamonds square, Thy gates are of right orient pearl Exceeding rich and rare! But what it could not do was to describe a condition in which the ordinary human being actively wanted to be.
Going into the details of Fun Theory helps you see that eudaimonia is complicated - that there are many properties which contribute to a life worth living. Which helps you appreciate just how worthless a galaxy would end up looking (with very high probability) if the galaxy was optimized by something with a utility function rolled up at random. The narrowness of this target is the motivation to create AIs with precisely chosen goal systems (Friendly AI).
Fun Theory is built on top of the naturalistic metaethics summarized in Joy in the Merely Good; as such, its arguments ground in "On reflection, don't you think this is what you would actually want (for yourself and others)?"
- The Fun Theory Sequence by Eliezer Yudkowsky describes some of the many complex considerations that determine what sort of happiness we most prefer to have - given that many of us would decline to just have an electrode planted in our pleasure centers.