The All-or-Nothing Pause Assumption: people assume that if we can't pause AI perfectly and forever, it's uselesss.

The All-or-Nothing Pause Assumption: people assume that if we can't pause AI perfectly and forever, it's uselesss.

Do you see how silly this is?

Imagine we applied this to biological weapons treaties.

1. If even just one weaponized superebola virus escapes a secret lab, humanity gets sent back to the dark ages.
2. At least one government will eventually defect and mess up.
3. Therefore, we shouldn't bother with any biological weapons treaties. We should let any corporation work full speed on developing and selling biological weapons to whoever they want.

This is preposterous.

Yes, treaties are impossible to enforce 100%

Yes, in certain industries, one mess up can lead to global catastrophes.

No, that doesn't mean all treaties are therefore useless.

Biological weapons development has been slown down massively by treaties making it so that private companies can't make them and that any government who wishes to do so must break their own ethical codes and do so in secret, under threat of extreme international sanctions if discovered.

Can you imagine what would have happened in some alternate timeline where we didn't ban biological weapons? 

Imagine the Bay Area developing all sorts of innovative new biological weapons and selling them on Amazon.

How long do you think humanity would have lasted?

We don't need a permanent and 100% effective AI pause for it to help humanity.

It will give safety researchers time to figure out how to build superintelligent AI safely.

It will give humanity time to figure out whether we want to create a new intelligent species, and if so, what values we would like to give them.

It will give us more time to live our terrible, wonderful, complicated monkey lives before the singularity changes everything.

So next time you see debates about pausing or slowing down AI, watch out for the All-or-Nothing Pause Assumption. Call it out when you see it.

Because something can be good without being perfect.