carado.moe

tammy's blog about
AI alignment, utopia, anthropics, and more;

also on twitter, lesswrong, rss, and at

is intelligence program inversion?

this is a thought i've had a while back, that i think i have enough notes on to make into a post. there's a sense in which running a program forwards is a trivial, "dumb" thing, but running it backwards — as in, given a result state, finding previous stats which would lead to it — is hard in a way that might exactly capture a reasonable notion of "intelligence".

here are some reasons to think this might make sense:

• the fact that intelligence entails explorings path, making choices, is captured in the fact that contrary to running a program forwards, running a program backwards is non-determinisic: a given result state may arise from one of multiple previous states, and this backwards-exploration of program-running might capture this aspect of intelligence.
• epistemology: what's epistemology, really? in its formal sense, as proposed in solomonoff induction, it's finding hypotheses that result in an observation, where hypotheses are programs. in this case, checking that a hypothesis leads to an observation is the easy "dumb" thing, whereas finding hypotheses for an observation is the difficult thing that takes intelligence.
• theorem proving: checking a proof is easy: just "dumbly" run the proof checker forwards. but, given the result "the checker outputs this theorem", finding a path that the proof checker would've followed to get there is equivalent to finding a proof of the theorem.
• maximizing utility: this one's a bit harder. supposedly, picking consequentially optimal actions given a goal is a matter of intelligence, so how does this apply here? one possible answer here is that picking optimal actions given observations is just epistemology + running the utility function forwards on each hypothesis, and comparing the results. but maybe shortcuting to epistemology is cheating here.