carado.moe

tammy's blog about
AI alignment, utopia, anthropics, and more;

my best alignment plan: QACI!

also on twitter, lesswrong, rss

what does it mean to value our survival?

let's say that i want to continue living, or that i want everyone currently on earth to continue living. what does that value look like, in a formalized format?

in my goal-program framework, i can see two ways to implement this:

• embedded: we make the goal-program contain either a full copy of myself/earth, or enough information to re-find it
• physical: we somehow make the values point to the real world — and then determine what counts as the thing and/or locate the thing we care about within it

some constraints apply:

• saving people who existed in the past — and maybe aliens outside of our light cone — seems like it would probly require resimulating earth; hence the embedded solution
• the physical solution might require significant philosophical work, or might be actually completely infeasable
• computational/storage limits might force us to stick to the physical solution

i believe there is some work towards the physical solution, so part the reason i tend to focus on the embedded solution is that it seems a lot less explored.