An EA had only $3 to give to anti-malarial bednets.

One day, she lost her $3.

Her EA group said, “I’m so sorry. That is so net negative. You must be so upset.”

The EA just said, “Maybe.”

A few days later, she found out her $3 had been stolen by a man living on less than a $1 a day, and it was basically a non-consensual GiveDirectly donation.

Her EA group said, “Congratulations! This is so net positive. You must be so happy!”

The EA just said, “Maybe.”

The poor man used his money to buy factory farmed chicken, causing far more suffering in the world.

Her EA group said, “I’m so sorry. This is so net negative. You must be so upset.”

The EA just said, “Maybe.”

The poor man, better nourished, was able to pull himself out of the poverty trap and work on AI safety, eventually leading to an aligned artificial superintelligence that ended all factory farming in the world.

Her neighbors said, “Congratulations! This is so net positive. You must be so happy!”

The EA just said, “Maybe.”

And it just keeps going.

Because consequentialism is the ethics of the gods. 

For we are but monkeys and cannot know the consequences of our actions.

Are deontology or virtue ethics the solution?

The EA just says, “Maybe.”

"Minimum Viable Coup" is my new favorite concept.
From Dwarkesh interviewing Paul Christiano, asking "what's the minimum capabilities needed for a superintelligent AI to overthrow the government?"

​If they're not conscious, we still have to worry about instrumental convergence. Viruses are dangerous even if they're not conscious.

But if they are conscious, we have to worry that we are monstrous slaveholders causing Black Mirror nightmares for the sake of drafting emails to sell widgets.

Of course, they might not care about being turned off. But there's already empirical evidence of them spontaneously developing self-preservation goals (because you can't achieve your goals if you're turned off).

​We had a whole class of people for ages who had nothing to do but hangout with people and attend parties. Just read any Jane Austen novel to get a sense of what it's like to live in a world with no jobs.

Only a small fraction of people, given complete freedom from jobs, went on to do science or create something big and important.

Most people just want to lounge about and play games, watch plays, and attend parties.

They are not filled with angst around not having a job.

In fact, they consider a job to be a gross and terrible thing that you only do if you must, and then, usually, you must minimize.

Our society has just conditioned us to think that jobs are a source of meaning and importance because, well, for one thing, it makes us happier.

We have to work, so it's better for our mental health to think it's somehow good for us.

And for two, we need money for survival, and so jobs do indeed make us happier by bringing in money.

Massive job loss from AI will not by default lead to us leading Jane Austen lives of leisure, but more like Great Depression lives of destitution.

We are not immune to that.

Us having enough is incredibly recent and rare, historically and globally speaking.

Remember that approximately 1 in 4 people don't have access to something as basic as clean drinking water.

You are not special.

You could become one of those people.

You could not have enough to eat.

So AIs causing mass unemployment is indeed quite bad.

But it's because it will cause mass poverty and civil unrest. Not because it will cause a lack of meaning.

(Of course I'm more worried about extinction risk and s-risks. But I am more than capable of worrying about multiple things at once)

​“The difference between nuclear arms treaties and AI treaties is that it’s so easy to copy AIs, so regulation is hopeless”

This is only true for existing models. Inventing new, state of the art models is incredibly difficult and expensive. It requires immense amounts of talent, infrastructure, money, compute, and innovations that people don’t yet know how to do.

Almost all of the human extinction risk from AIs come from not-yet-invented superintelligent AI models.

North Korea or a terrorist group cannot just defect from an AI treaty and build superintelligent AI.

And it’s relatively straightforward to monitor and prevent the amount of compute necessary to make a superintelligent AI (e.g. monitoring electrical grids, specialized GPUs, satellite imagery, etc)

Once it’s already invented, then yes, people could easily steal it.

But if we just stop sometime 𝘣𝘦𝘧𝘰𝘳𝘦 we have superintelligent AI, then it will be very hard for any group to defect.

Also, by the time we superintelligent AI, it’s probably already too late, and it will be up to the superintelligence what to do, not humans anymore.

Do you live in Berkeley? 

Are you feeling particularly anxious or depressed? 

Consider moving to SF or elsewhere. 

I've heard from so many people that the AI safety community in Berkeley is particularly damaging to mental health. 

I've only felt depressed about AI safety once & it was when I was in Berkeley for a few weeks. It went away after I left. 

From what I've heard it seems to be from a mix of: 

• Lots of gossip and anxiety about results of said gossip, which often lead to losses of invitations, funding, and job opportunities
  
• A gloomy vibe, where a critical mass of people think it's an incredibly high chance of doom
  
• Escalating purity testing cycles

Remember: you're the average of the 5 people you spend the most time with. 

You might be depressed or anxious because of objective facts about the world. 

But you might also be depressed or anxious because you're surrounded by depressed or anxious people

I've heard good things about the SF AI safety community. London's EA community was also far less neurotic in my experience than many others. Felt much more balanced and less insular.

Disclaimer: this will only work for a subset of you. Law of Equal and Opposite Advice and all that. It might only even work for me. This definitely feels like a weird psychological trick that might only work with my brain. 

I spent my twenties being absolutely devastated by uncertainty. I saw the suffering in the world and I desperately wanted to help, but the more I learned and the more I tried, the wider my confidence intervals got. 

Maybe I could promote bednets. But what about the meat eater problem?

Maybe I could promote veganism? But what about the small animal replacement problem? 

Even giving out free hugs (the most clearly benign thing I could think of) might cause unexpected trauma for some unknown percentage of the population such that it negates all the positives.

It eventually reached a crescendo in 2020 where I sunk into absolute epistemic hopelessness. An RCT had just been published about the intervention I was doing that didn't even show that the intervention didn't work. It was just ambiguous. If at least it had been obviously zero impact, I could have moved on. But it was ambiguous for goodness sake! 

I actually briefly gave up on altruism. 

I was going to go be a hippie in the woods and make art and do drugs. After all, if I couldn't know if what I was doing was helping or even hurting, I might as well be happy myself. 

But then…. I saw something in the news about the suffering in the world. And I wanted to help. 

No, a part of me said. You can't help, remember? Nothing works. Or you can never tell if it's working. 

And then another thing showed up in my social media feed…. 

But no! It wasn’t worth trying because the universe was too complex and I was but a monkey in shoes. 

But still. . . . another part of me couldn’t look away. It said “Look at the suffering. You can’t possibly see that and not at least try.” 

I realized in that moment that I couldn’t actually be happy if I wasn’t at least trying. 

This led to a large breakthrough in how I felt. Before, there was always the possibility of stopping and just having fun. So I was comparing all of the hard work and sacrifice I was doing to this ideal alternative life. 

When I realized that even if I had basically no hope, I’d still keep trying, this liberated me. There was no alternative life where I wasn’t trying. 

It felt like the equivalent of burning the ships. No way to go but forward. No temptation of retreat. 

Many things aren’t bad in and of themselves, but bad compared to something else. If you remove the comparison, then they’re good again. 

But it wasn’t over yet. I was still deeply uncertain. I went to Rwanda to try to actually get as close to ground truth as possible, while also reading a ton about meta-ethics, to get at the highest level stuff, then covid hit. 

While I was stuck in lockdown, I realized that I should take the simulation hypothesis seriously. 

You’d think this would intensify my epistemic nihilism, but it didn’t.

It turned me into an epistemic absurdist.

Which is basically the same thing, but happy. 

Even if this is base reality, I’m profoundly uncertain about whether bednets are even net positive. 

Now you add that this might all be a simulation?!? 

For real?! 

(Pun was unintentional but appreciated, so I’m keeping it) 

This was a blessing in disguise though, because suddenly it went from:

1. “If you make choice A a baby will die and it’s on your hands” to 
2. “If you make choice A, you’ll never really know if it helps or hurts due to deep massive uncertainty, but hey, might as well try”

The more certain you feel, the more you feel you can control things, and that leads to feeling more stressed out. 

As you become more uncertain, it can feel more and more stressful, because there’s an outcome you care about and you’re not sure how to get there. 

But if you have only very minimal control, you can either freak out more, because it’s out of your control, or you can relax, because it’s out of your control. 

So I became like the Taoist proverb: "A drunkard falls out of a carriage but doesn't get hurt because they go limp."

If somebody walked by a drowning child that would be trivially easy to save, I’d think they were a monster. 

If somebody walks by a deeply complex situation where getting involved may or may not help and may even accidentally make it worse, but then tries to help anyway, I think they’re a good person and if it doesn’t work out, well, hey, at least they tried. 

I relaxed into the uncertainty. The uncertainty means I don’t have to be so hard on myself, because it’s just too complicated to really know one way or the other. 

Nowadays I work in AI safety, and whenever I start feeling anxious about timelines and p(doom), the most reliable way for me to feel better is to remind myself about the deep uncertainty around everything. 

“Remember, this might all be a simulation. And even if it isn’t, it’s really hard to figure out what’s net positive, so just do something that seems likely to be good, and make sure it’s something you at least enjoy, so no matter what, you’ll at least have had a good life”

How can other people apply this?

I think this won’t work for most people, but you can try this on and see if it works for you:

1. Imagine the worst, and see if you’d still try to help. Imagine you’re maximally uncertain. If you’d still try to help in this situation, you can feel better, knowing that no matter what, you’ll still care and do your best. 
2. Relax into the uncertainty. Recognize that you shouldn’t be too hard on yourself, because there aren't actually just drowning babies needing a simple lift. 

Anyways, while I’m sure this won’t work for most people, hopefully some people who are currently struggling in epistemic nihilism might be able to come out the other side and enjoy epistemic absurdism like me. 

But in the end, who knows? 

You can't just say “epistemic status: garbage” and then hope your words don’t have consequences

Especially online, where everything you write is forever and can be damaging a person’s reputation, mental health, and ability to do good until the singularity.

This isn’t to say you shouldn’t criticise. But you should publicly criticize when you have decent evidence and arguments, not when it’s just hearsay, vibe, anonymous sources, or any other form of terrible epistemics.

1. I put high odds (~80%) that there will be a warning shot that’s big enough that a pause becomes very politically tractable (~75% pause passed, conditional on warning shot).
2. The supply chain is brittle, so people can unilaterally slow down development. The closer we get, more and more people are likely to do this. There will be whack-a-mole, but that can give us a lot of time.
3. We’ve banned certain technological development in the past, so we have proof of concept.
4. We all don’t want to die. This is something of virtually all political creeds can agree on.

​​*Definition of a pause for this conversation: getting us an extra 15 years before ASI. So this could either be from a international treaty or simply slowing down AI development