smaller X-risk

posted on 2022-05-16

a superintelligence killing us all is a superintelligent, very large X-risk.

the superintelligence will tile its values in all directions; not just through space at the speed of light or faster, but also, if it can, by hacking physics and traversing across, for example, worldlines of the quantum many-worlds.

we may be able to create smaller X-risks, that only make us extinct in this timeline, on this earth. there are a few reasons we may want to do this:

other timelines might have a better shot than us, and us booting a superintelligence may reduce their chances through weird stuff like intertimeline hacking
to avoid S-risks, including S-risks that may be involved in instrumental cosmic-scale X-risk (maybe superintelligence wants to simulate civilizations in various ways for acausal trade or other acausal weirdness reasons?)
the next intelligent species on earth is more likely than us to solve alignment before superintelligence, and seems likely enough to be at least a little bit aligned with us (better than cosmic X-risk, at least)
same as above, but for nearby aliens (whether current or future)

smaller X-risk, where we limit damage to just our civilization, seems harder than tiling the cosmos with paperclips; but at least it might be easier than other plans.

in a similar way, reducing our civilization to ashes without actually becoming extinct might also be a way to get another shot, if we think we're likely to do less badly next time.

remember: this is bigger than all of us. when the fate of the cosmos is at play, we can't afford to be too selfish.

posted on 2022-05-16