The utility function is not up for grabs
The constraints of decision theory are only there to help you win; they don't specify what constitutes a win. Rationality in and of itself cannot constrain what you want, except insofar as what you thought you wanted was logically contradictory and couldn't exist anyway. Hence the saying: the utility function is not up for grabs.
- Newcomb's Problem and Regret of Rationality
- Something to Protect
- Rationality is Systematized Winning