"Minimum Viable Coup" is my new favorite concept.
From Dwarkesh interviewing Paul Christiano, asking "what's the minimum capabilities needed for a superintelligent AI to overthrow the government?"

​If they're not conscious, we still have to worry about instrumental convergence. Viruses are dangerous even if they're not conscious.

But if they are conscious, we have to worry that we are monstrous slaveholders causing Black Mirror nightmares for the sake of drafting emails to sell widgets.

Of course, they might not care about being turned off. But there's already empirical evidence of them spontaneously developing self-preservation goals (because you can't achieve your goals if you're turned off).

​We had a whole class of people for ages who had nothing to do but hangout with people and attend parties. Just read any Jane Austen novel to get a sense of what it's like to live in a world with no jobs.

Only a small fraction of people, given complete freedom from jobs, went on to do science or create something big and important.

Most people just want to lounge about and play games, watch plays, and attend parties.

They are not filled with angst around not having a job.

In fact, they consider a job to be a gross and terrible thing that you only do if you must, and then, usually, you must minimize.

Our society has just conditioned us to think that jobs are a source of meaning and importance because, well, for one thing, it makes us happier.

We have to work, so it's better for our mental health to think it's somehow good for us.

And for two, we need money for survival, and so jobs do indeed make us happier by bringing in money.

Massive job loss from AI will not by default lead to us leading Jane Austen lives of leisure, but more like Great Depression lives of destitution.

We are not immune to that.

Us having enough is incredibly recent and rare, historically and globally speaking.

Remember that approximately 1 in 4 people don't have access to something as basic as clean drinking water.

You are not special.

You could become one of those people.

You could not have enough to eat.

So AIs causing mass unemployment is indeed quite bad.

But it's because it will cause mass poverty and civil unrest. Not because it will cause a lack of meaning.

(Of course I'm more worried about extinction risk and s-risks. But I am more than capable of worrying about multiple things at once)

​“The difference between nuclear arms treaties and AI treaties is that it’s so easy to copy AIs, so regulation is hopeless”

This is only true for existing models. Inventing new, state of the art models is incredibly difficult and expensive. It requires immense amounts of talent, infrastructure, money, compute, and innovations that people don’t yet know how to do.

Almost all of the human extinction risk from AIs come from not-yet-invented superintelligent AI models.

North Korea or a terrorist group cannot just defect from an AI treaty and build superintelligent AI.

And it’s relatively straightforward to monitor and prevent the amount of compute necessary to make a superintelligent AI (e.g. monitoring electrical grids, specialized GPUs, satellite imagery, etc)

Once it’s already invented, then yes, people could easily steal it.

But if we just stop sometime 𝘣𝘦𝘧𝘰𝘳𝘦 we have superintelligent AI, then it will be very hard for any group to defect.

Also, by the time we superintelligent AI, it’s probably already too late, and it will be up to the superintelligence what to do, not humans anymore.

Do you live in Berkeley? 

Are you feeling particularly anxious or depressed? 

Consider moving to SF or elsewhere. 

I've heard from so many people that the AI safety community in Berkeley is particularly damaging to mental health. 

I've only felt depressed about AI safety once & it was when I was in Berkeley for a few weeks. It went away after I left. 

From what I've heard it seems to be from a mix of: 

• Lots of gossip and anxiety about results of said gossip, which often lead to losses of invitations, funding, and job opportunities
  
• A gloomy vibe, where a critical mass of people think it's an incredibly high chance of doom
  
• Escalating purity testing cycles

Remember: you're the average of the 5 people you spend the most time with. 

You might be depressed or anxious because of objective facts about the world. 

But you might also be depressed or anxious because you're surrounded by depressed or anxious people

I've heard good things about the SF AI safety community. London's EA community was also far less neurotic in my experience than many others. Felt much more balanced and less insular.

Disclaimer: this will only work for a subset of you. Law of Equal and Opposite Advice and all that. It might only even work for me. This definitely feels like a weird psychological trick that might only work with my brain. 

I spent my twenties being absolutely devastated by uncertainty. I saw the suffering in the world and I desperately wanted to help, but the more I learned and the more I tried, the wider my confidence intervals got. 

Maybe I could promote bednets. But what about the meat eater problem?

Maybe I could promote veganism? But what about the small animal replacement problem? 

Even giving out free hugs (the most clearly benign thing I could think of) might cause unexpected trauma for some unknown percentage of the population such that it negates all the positives.

It eventually reached a crescendo in 2020 where I sunk into absolute epistemic hopelessness. An RCT had just been published about the intervention I was doing that didn't even show that the intervention didn't work. It was just ambiguous. If at least it had been obviously zero impact, I could have moved on. But it was ambiguous for goodness sake! 

I actually briefly gave up on altruism. 

I was going to go be a hippie in the woods and make art and do drugs. After all, if I couldn't know if what I was doing was helping or even hurting, I might as well be happy myself. 

But then…. I saw something in the news about the suffering in the world. And I wanted to help. 

No, a part of me said. You can't help, remember? Nothing works. Or you can never tell if it's working. 

And then another thing showed up in my social media feed…. 

But no! It wasn’t worth trying because the universe was too complex and I was but a monkey in shoes. 

But still. . . . another part of me couldn’t look away. It said “Look at the suffering. You can’t possibly see that and not at least try.” 

I realized in that moment that I couldn’t actually be happy if I wasn’t at least trying. 

This led to a large breakthrough in how I felt. Before, there was always the possibility of stopping and just having fun. So I was comparing all of the hard work and sacrifice I was doing to this ideal alternative life. 

When I realized that even if I had basically no hope, I’d still keep trying, this liberated me. There was no alternative life where I wasn’t trying. 

It felt like the equivalent of burning the ships. No way to go but forward. No temptation of retreat. 

Many things aren’t bad in and of themselves, but bad compared to something else. If you remove the comparison, then they’re good again. 

But it wasn’t over yet. I was still deeply uncertain. I went to Rwanda to try to actually get as close to ground truth as possible, while also reading a ton about meta-ethics, to get at the highest level stuff, then covid hit. 

While I was stuck in lockdown, I realized that I should take the simulation hypothesis seriously. 

You’d think this would intensify my epistemic nihilism, but it didn’t.

It turned me into an epistemic absurdist.

Which is basically the same thing, but happy. 

Even if this is base reality, I’m profoundly uncertain about whether bednets are even net positive. 

Now you add that this might all be a simulation?!? 

For real?! 

(Pun was unintentional but appreciated, so I’m keeping it) 

This was a blessing in disguise though, because suddenly it went from:

1. “If you make choice A a baby will die and it’s on your hands” to 
2. “If you make choice A, you’ll never really know if it helps or hurts due to deep massive uncertainty, but hey, might as well try”

The more certain you feel, the more you feel you can control things, and that leads to feeling more stressed out. 

As you become more uncertain, it can feel more and more stressful, because there’s an outcome you care about and you’re not sure how to get there. 

But if you have only very minimal control, you can either freak out more, because it’s out of your control, or you can relax, because it’s out of your control. 

So I became like the Taoist proverb: "A drunkard falls out of a carriage but doesn't get hurt because they go limp."

If somebody walked by a drowning child that would be trivially easy to save, I’d think they were a monster. 

If somebody walks by a deeply complex situation where getting involved may or may not help and may even accidentally make it worse, but then tries to help anyway, I think they’re a good person and if it doesn’t work out, well, hey, at least they tried. 

I relaxed into the uncertainty. The uncertainty means I don’t have to be so hard on myself, because it’s just too complicated to really know one way or the other. 

Nowadays I work in AI safety, and whenever I start feeling anxious about timelines and p(doom), the most reliable way for me to feel better is to remind myself about the deep uncertainty around everything. 

“Remember, this might all be a simulation. And even if it isn’t, it’s really hard to figure out what’s net positive, so just do something that seems likely to be good, and make sure it’s something you at least enjoy, so no matter what, you’ll at least have had a good life”

How can other people apply this?

I think this won’t work for most people, but you can try this on and see if it works for you:

1. Imagine the worst, and see if you’d still try to help. Imagine you’re maximally uncertain. If you’d still try to help in this situation, you can feel better, knowing that no matter what, you’ll still care and do your best. 
2. Relax into the uncertainty. Recognize that you shouldn’t be too hard on yourself, because there aren't actually just drowning babies needing a simple lift. 

Anyways, while I’m sure this won’t work for most people, hopefully some people who are currently struggling in epistemic nihilism might be able to come out the other side and enjoy epistemic absurdism like me. 

But in the end, who knows? 

You can't just say “epistemic status: garbage” and then hope your words don’t have consequences

Especially online, where everything you write is forever and can be damaging a person’s reputation, mental health, and ability to do good until the singularity.

This isn’t to say you shouldn’t criticise. But you should publicly criticize when you have decent evidence and arguments, not when it’s just hearsay, vibe, anonymous sources, or any other form of terrible epistemics.

1. I put high odds (~80%) that there will be a warning shot that’s big enough that a pause becomes very politically tractable (~75% pause passed, conditional on warning shot).
2. The supply chain is brittle, so people can unilaterally slow down development. The closer we get, more and more people are likely to do this. There will be whack-a-mole, but that can give us a lot of time.
3. We’ve banned certain technological development in the past, so we have proof of concept.
4. We all don’t want to die. This is something of virtually all political creeds can agree on.

​​*Definition of a pause for this conversation: getting us an extra 15 years before ASI. So this could either be from a international treaty or simply slowing down AI development

𝐍𝐞𝐰 𝐦𝐞𝐧𝐭𝐚𝐥 𝐡𝐞𝐚𝐥𝐭𝐡 𝐩𝐫𝐨𝐠𝐫𝐚𝐦 𝐟𝐨𝐫 𝐩𝐞𝐨𝐩𝐥𝐞 𝐰𝐨𝐫𝐤𝐢𝐧𝐠 𝐨𝐧 𝐀𝐈 𝐬𝐚𝐟𝐞𝐭𝐲! 🥳

It’s not therapy. It’s what I wish therapy was, but totally isn’t.

It’s a short program that lasts 4-12 weeks, where you systematically try 5-30 techniques until you find something that fixes an emotional problem you're struggling with (e.g. anxiety, impostor syndrome, low mood, etc).

Here’s how it works:

𝐅𝐢𝐫𝐬𝐭 𝐜𝐚𝐥𝐥: 𝐮𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝 𝐚𝐧𝐝 𝐩𝐥𝐚𝐧
​

1. 𝐔𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝. Discuss the problem and generate at least 5 hypotheses for what’s causing the problem.
2. 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐞 𝐩𝐨𝐬𝐬𝐢𝐛𝐥𝐞 𝐬𝐨𝐥𝐮𝐭𝐢𝐨𝐧𝐬. In this process, we’ll generate and look at least 30 possible ways to fix the problem. This will come from a wide swathe of possible solutions, such as CBT, internal family systems, problem-solving, changing your environment, meditation, diet, exercise, supplements, loving-kindness, gratitude, stoicism, improving sleep hygiene, checking for nutritional deficiencies, bibliotherapy, meds (not prescribed by me, but I might recommend you talk to a doctor), etc.
3. 𝐏𝐫𝐢𝐨𝐫𝐢𝐭𝐢𝐳𝐞 𝐩𝐨𝐬𝐬𝐢𝐛𝐥𝐞 𝐬𝐨𝐥𝐮𝐭𝐢𝐨𝐧𝐬. We’ll prioritize the possible solutions by their probability of working, how costly they are to try, whether they can be tried with things in parallel, etc.
4. 𝐏𝐥𝐚𝐧 𝐭𝐨 𝐢𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭 𝟓-𝟏𝟎 𝐨𝐟 𝐭𝐡𝐞 𝐩𝐨𝐬𝐬𝐢𝐛𝐥𝐞 𝐬𝐨𝐥𝐮𝐭𝐢𝐨𝐧𝐬 𝐨𝐯𝐞𝐫 𝐭𝐡𝐞 𝐧𝐞𝐱𝐭 𝟏-𝟑 𝐰𝐞𝐞𝐤𝐬. This will include things like setting up accountability systems, commitment devices, time blocking, etc. It will all be customized to what works best for you and for the particular techniques.
   
   

𝐍𝐞𝐱𝐭: 𝐢𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 𝐚𝐧𝐝 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 𝐜𝐲𝐜𝐥𝐞

You’ll spend the next 1-3 weeks actually putting the most promising techniques into practice.

You’ll keep track of your symptoms. If your symptoms go away, then we’ll analyze what happened. Sometimes it’ll be obvious what’s helping, and you can just keep doing that thing. If not, then we can start remove the techniques one at a time. If the symptoms come back, then we just bring back the technique that we removed, and we know what was doing the magic.

Experimenting in parallel means you get to feel better sooner and continue to feel good while we figure out what the problem was. 

If your symptoms don’t go away after 1-2 weeks, then we’ll prioritize the next 5-10 techniques to try.

This process will happen up to 3 times.

By the end, you’ll have either resolved your issues, or you’ll at least have tried ~30 techniques to fix the problem. Even if you haven’t, you’ll probably have found at least a few more techniques to add to your repertoire of things that you enjoy.

Apply here

𝐈𝐬 𝐭𝐡𝐢𝐬 𝐭𝐡𝐞𝐫𝐚𝐩𝐲?

It’s not therapy. It’s what I wish therapy was, but totally isn’t. 

𝐄𝐦𝐨𝐭𝐢𝐨𝐧𝐚𝐥 𝐩𝐫𝐨𝐛𝐥𝐞𝐦𝐬 𝐭𝐡𝐞 𝐩𝐫𝐨𝐠𝐫𝐚𝐦 𝐜𝐚𝐧 𝐡𝐞𝐥𝐩 𝐰𝐢𝐭𝐡:

Stress
Impostor syndrome
Burnout
Anxiety
Hopelessness
Feeling overwhelmed 
Depression (mild or moderate. Not severe)
Self-esteem issues
Motivation issues
Numbness
Sadness
Work life balance
Guilt
Sleep issues
Loneliness
Existential angst
Perfectionism
Relationship problems

𝐄𝐦𝐨𝐭𝐢𝐨𝐧𝐚𝐥 𝐩𝐫𝐨𝐛𝐥𝐞𝐦𝐬 𝐭𝐡𝐞 𝐩𝐫𝐨𝐠𝐫𝐚𝐦 𝙘𝙖𝙣𝙣𝙤𝙩 𝐡𝐞𝐥𝐩 𝐰𝐢𝐭𝐡 

Suicidality
Bipolar
ADHD
Gender dysphoria
Anger management
Substance use disorders
Autism related emotional issues
Cluster B personality disorders (e.g. BPD, APD, HPD, NPD)
Anything where you're experiencing psychosis
Anything where you're experiencing paranoia or delusions

𝐇𝐨𝐰 𝐦𝐮𝐜𝐡 𝐝𝐨𝐞𝐬 𝐢𝐭 𝐜𝐨𝐬𝐭?

It's free if you:

• Work in AI safety
• Are currently unemployed, but usually work in AI safety
• Earn to give and give >$50,000 a year to AI safety

I'm offering this service for free because mental health is one of the main blockers to people having an impact in AI safety. 

I think x-risks and s-risks from AI are the most important things to work on. And I'm good at emotional problem-solving. So if I help people working on AI safety be happier, then I'm helping make sure AI doesn't kill everybody. 

Timelines are too short to work with people who are not working in AI safety, but since I'm a rationalist and everybody has their price, I would do it for a non-AI safety person if they donated $10,000 or more to Nonlinear or an AI safety org working on pausing or slowing down AI development. 

Apply here

I have very limited time, so can only take on a small fraction of clients who apply. If you do not get in, I recommend checking out this vetted list of therapists or this compilation of mental health techniques for dealing with AI safety. 

• Mentnav
• Lesswrong