posted on 2022-12-11 — also cross-posted on lesswrong, see there for comments

a rough sketch of formal aligned AI using QACI

in this post, i put forth some of my current thoughts about the shape of a formal aligned AI using QACI for its decision — "decision" in the singular here, as this is sufficient when the AI's decision can be "run me again but with these diffirent inputs". as it turns out, this doesn't require solving as many things as i'd thought — it seems like QACI might be general enough to delegate picking a decision theory and solving embedded agency to the counterfactual consideration of the past user.

we'll posit:

we'll define the following:

a "carver" function C:W×2(×W)××(W) which returns a set of tuples of:

this is done by splitting a world w into the piece of data x, and "everything else", denoted tw\x. in practice with arbitrary other worlds, tx would return "garbage" the immense majority of the time, but the hope is that given a same carving txq,twq for the question q, a same carving tqr,twr for the answer would work enough time to give a signal that would tend to beat the overall noise of the failing cases.


we'll define QACI:W×××2×, the question-answer counterfactual interval device used to consider answers to counterfactual questions q, given a world hypothesis w and a known question q and answer r:


note how K measures the simplicity of all four t functions together so as to favor them being simple but also similar, but ignores the simplicity of the "rest of the world" tw\x values..

finally, we can define our AI G:××A as a function of q,r but also an observation o which could, in practice, be anything that lets the AI as well as the user better locate themselves in the set of possible worlds.


what's going on here, is for each world hypothesis w weighed by its simplicity K(w) we're finding carvings for q and r that let us inject a counterfactual question and get a counterfactual answer, favoring simpler carvings using α. the counterfactual rerun of the past user is given the world hypothesis w being considered, as well as the action a — we're not giving the user q,r,o because that would make the counterfactual question guaranteed to be strictly larger than the original question, which seems like it might break things, and is not absolutely necessary. the user gets to return an arbitrary function to be evaluated, to which q,r,o are actually passed. we are, however, clamping the result of r inside [0;1] and then multiplying it by K(w)α ourselves, so as to avoid erroneous carvings getting to influence the decision process too much by returning arbitrary large values. the hope here is merely that desired carvings generally outweigh undesired carvings, and especially adverserial carvings.

ways this could go wrong include:

this is of course highly uncomputable. the intent here, is to use something like logical induction to approximate good results to this function. what makes me hopeful that a powerful AI can make helpful guesses as to what actions this process would find, if it is indeed aligned, is that even i, a mere human mind, feel like i can make some helpful guesses as to what actions this process would find.

posted on 2022-12-11 — also cross-posted on lesswrong, see there for comments

CC_ -1 License unless otherwise specified on individual pages, all posts on this website are licensed under the CC_-1 license.
unless explicitely mentioned, all content on this site was created by me; not by others nor AI.