With @luke_drago_, I’m cofounding Workshop Labs, a public benefit corporation preventing human disempowerment from AI. See below for:.-impact case.-what we’re building.-what we hope the future looks like.-what we’re hiring for.

We’re hiring for people who can accelerate us in building the ML infrastructure and doing the R&D for our models. We have promising initial results, but we’ll need to move exceptionally fast. If you buy this mission, apply! (Company website link in my profile).

One concern is misuse risks from boosting bad actors. We’re committed to both safety and verifiable user data confidentiality. We’re building safeguards against illegal or societally-dangerous capabilities, that (like all our infra!) never let us see user data, even if we tried.

Further down the line, this also gives us an interesting primitive for bottom-up governance & coordination. Just as one example, you can imagine your model advocating for you & your worldview in an online parliament with millions of other models.

This also means everyone can have a model aligned specifically to them. Alignment to a specific user & inter-user-disagreements being handled by normal societal/political means seems more robust than having a small set of people decide one target that every AI is aligned to.

In addition to preventing economic disempowerment, this is also a much more pluralistic and human-involved vision of how the AI economy could work than the single superintelligent AI sovereign. I think this importantly reduces gradual disempowerment & goodharting risks.

At first, your model will help you with those parts of your work that you have demonstrated many times, but ChatGPT is too generic to do. Later you might be like a CEO managing models. But you’ll stay involved, and own it.

Right now, the people have the data and the labs have the AI. Will the labs get the data before the people get the AI? Top-down automation and disempowerment, or bottom-up automation and at least a starting toehold of relevance going into the AI revolution?.

But: people exhibit those skills, tacit knowledge, etc. every day. Why not let people take this personal data moat, and harness it to train their own AI that works with them?. Ofc, data+model needs to be inaccessible to anyone but the person. We think we know how to build this.

Concretely: AIs are improving much slower at hard-to-measure, local-info-requiring, tacit-knowledge-based tasks; big data bottlenecks remain. Default course: labs spend lots of effort on datasets & environments, automate industry by industry, outcompete everyone else.

How do you keep people economically relevant as superintelligent AI approaches? As outlined in The Intelligence Curse, abstractly we want:.1: Keep humans in the loop of the creation of value.2: Decentralized ownership & control over AI.And also: better institutions & governance.

In the Intelligence Curse, Luke & I also argued at length that humans remaining part of the economy matters. You need to solve for the equilibrium: if economic & military power is all AI, then power is divorced from humans, and it’s much less likely that power will serve humans.

Some good starting points for approaching these problems:.1: top-down solutions are hard; trust in experimentation, plurality, people thinking & trying for themselves, liberalism broadly.2: human value is fragile; you need humans around & in charge & deciding.

Solving the more systemic/societal/economic issues is hard. Luke & I spent a lot of time thinking (resulting in The Intelligence Curse & other writings, and I have a few more posts coming up on the more philosophical side), before having an idea for how to tackle them robustly.

I see a place where we’re collectively dropping the ball. Most AI risk work aims at single-single alignment & technical risk. But: gradual disempowerment, the intelligence curse, going-out with-a-whimper. Attached is a screenshot from a long-unfinished draft of mine:

The company is fundamentally about a mission. We think anyone working in AI has a choice to make about what future they’re advancing, and that this choice matters, especially given the potential power and lock-in effects of AI.

some highly neglected interventions here:.

Great eval! Benchmarks give surprisingly little insight into actual model competence in the wild. Running a vending machine does.

More ideas here: .

RT @NinaPanickssery: I am becoming increasingly convinced that the key to safe AI is to delegate as much of the goal/value-setting to human….