RT @MatthewFdashR: At least for me, the big-picture motivation behind our RLC paper is a research vision for scalable AI alignment via mini….

RT @plain_simon: @aryaman2020 @cloneofsimo Yes and there are good reasons to expect the functions we want to learn have this property and t….

RT @_arohan_: An explanation of Match3 functions and motivation for 2-simplicial attention. You have a shelf of about 30 film which includ….

RT @JJitsev: Using scaling laws to compare standard pairwise and higher order triple interaction attention is nice. However, I think in cur….

RT @AnimaAnandkumar: I have been advocating tensor methods for almost decade and a half. Take a look at our tensor methods in deep learning….

RT @nsaphra: 🚨 New preprint! 🚨 Phase transitions! We love to see them during LM training. Syntactic attention structure, induction heads, g….

RT @l_spermatikos: It's strangely satisfying how knot theory has looped back around on itself. It was Lord Kelvin's proposal that atoms wer….

RT @Turn_Trout: The "sleeper agent" terminology is hyperbolic and unfortunate IMO. Crying wolf. Should have reserved such an aggressive tit….

RT @bschne: what the. 🤯

RT @plain_simon: Great discussion on the big picture of singular learning theory and its applications to interpretability of neural network….

RT @baophamhq: Diffusion models create novel images, but they can also memorize samples from the training set. How do they blend stored fea….

RT @plain_simon: Last week I gave a talk in the. Belgian-Dutch Junior algebraic geometry seminar. on the real log-canonical threshold in s….

RT @geoffreyirving: New alignment theory paper! We present a new scalable oversight protocol (prover-estimator debate) and a proof that hon….

RT @neuro_kim: New work on relational reasoning in transformers!. TLDR: In-Weight and In-Context Learning inductive baises are really diffe….

As far as I know this doesn't yet have a very strong theoretical basis (we are working on it). I think sloppy reasoning about simplicity biases is having a kind of hidden systematic influence on people's thinking about various approaches to alignment.

However it isn't hard to imagine than in a multi-agent system with a long-horizon task, actually \pi is simpler because it is more robust to interventions by other agents. The savant who sits in the corner and builds a tower (this is X) can be bullied.

A common factor in many people's thoughts about alignment is the idea that a policy \pi which does task X and also seeks influence is more complex than a policy \psi which just tries to do X. Thus if we "keep the optimisation pressure on" we'll discourage influence-seeking. 🧵.

RT @DavidDuvenaud: What to do about gradual disempowerment? We laid out a research agenda with all the concrete and feasible research proje….

RT @ChrSzegedy: Why is verified superintelligence the next logical step?. What problems will it solve and how?. 🧵 1/7.

RT @SimonShaoleiDu: EM is a classic, but it can fail even for 3-component Gaussian mixtures. Why is EM wide used? Over-parameterization!.We….