carado.moe

tammy's blog about
AI alignment, utopia, anthropics, and more;

also on twitter, lesswrong, rss, and at

bracing for the alignment tunnel

it looks like we're gonna invent AI that kills everyone before we figure out AI alignment.

what this means is that soon, if not already, we are going to start bleeding timelines, hard; by which i mean, an increasing ratio of multiverse-instants are gonna become dominated by unaligned AIs — and thus be devoid of population (probably).

after that, there is a period in the (ever-diminishing amount of) surviving timelines, where we ride on quantum immortality to solve alignment; after which, we finally reach U-lines, hopefully.

by many theories of anthropics, observing existing either before or after that period is a lot more likely than observing existing in it. before the period, it is more likely because there are a lot more populated timelines in which to exist; after the period, it is more likely because we can hopefully "repopulate horizontally" by allowing the population to increase again.

if i am correct in the reasoning in this post, then being someone who exists in this very narrow alignment "tunnel" is exceedingly unlikely (barring weird circumstances such as post-singularity mankind choosing to simulate many variants of the tunnel for some reason). indeed, if you do observe being in it, you should think that something weird is going on, and update against the narrative presented in this post.

yet, again if i am correct, this is a period where we need to hold tight and work on alignment, perhaps as quickly as possible in order to reduce astronomical waste. very few us's inhabit the tunnel, but those very few us's are the critical ones who we need to care about.

so we need to brace our minds for the alignment tunnel. we need to commit to be persons who, if we observe being in the tunnel, will keep working on alignment even if, from inside those timelines, it looks like the reasoning i'm presenting here can't possibly be right. this is perhaps a weird case of instrumental rationality.

(note that i'm not saying the conclusion of observing being in those timelines should be to stop working on alignment; perhaps we would want to work on it either way, in which case we don't have to worry about anything. but i worry that it could lead us to other places such as "oh, maybe this AI killing everyone business isn't real after all, or maybe a weird alien force is preventing us from dying somehow")