carado.moe

tammy's blog about
AI alignment, utopia, anthropics, and more;

also on twitter, lesswrong, rss, and at

do not hold on to your believed intrinsic values — follow your heart!

i posit the following framework for thinking about intrinsic values (hereby just called "values").

somewhere out there, there is your value system.

it is a function (in the mathematical sense) that takes as input things, and spits out feelings. that function is where your true values lie; they are that function.

how is that function encoded in your brain? who knows! i don't have an answer, and there may not be an answer.

in conscious thought, however, you don't have access to the source code of that function, whatever it is. the best we can do for the moment seems to be to try thinking about it real hard, throw various things at it and examine the output (perhaps through some mildly systematic process), and define another function that tries to approximate what your actual values are. this is hard and takes a lot of work. perhaps someday we will have a device that can scan your brain and give you a better idea of what your value function looks like, but at the moment that is not the case.

so, you build up an idea of what your values might be. here is where i think a lot of people make a mistake: they choose to believe strongly that this guess is their actual set of values (even though it likely isn't). they crystallize those values; they live by them, until they in turn become influenced by those values and perhaps actually adopt them. (the actual value function is mutable!)

this is generally bad; you should want to preserve whatever your values are. hence the appeal that stands as the title of this post: do not hold on to the approximate function that is your best guess at what your value system is; you're only human, your guess is likely incorrect, and adopting it would run the risk of damaging your actual values; a function which, while hard to figure out, can be mutable, will certainly be mutated by acting as if your values are not what they are, and whose mutation you should generally want to avoid.

so, pay attention to your feelings. they are what is the output of your actual values system, by definition; follow your heart, not your reason's best guess.

note that this is not an appeal to favor deontology over consequentialism: how you feel can be about actions (deontology) or about outcomes (consequentialism), and which one it is is orthogonal to whether you follow that system, or whether you decide to follow your current best approximation of it. if you are consequentialist (as i recommend), just make sure to give your value system a full picture of what the outcome would look like, and then decide based on what feelings are produced by that.

meta note: this framework for thinking about values should be itself held with suspicion, as should probly any formal framework that concerns values. you should be careful about holding on to it just like you should have been careful about holding onto your believed values (careful enough to be able to consider the present post, for example). which isn't to say don't believe what i just wrote, but leave room for it to be wrong, partially or wholly.