AI safety’s most underrated fic: Seed.

AI safety’s most underrated fic: Seed.

​Starts off with a discussion about why they’re (trying to) shut down a supertintelligent AI.

“Why? What are you afraid of? Viruses?”

“No. We’re afraid of accidentally creating a merciless god.”

“You guys can really do that? Why would you even do that?”  
​
It also starts off by talking about interpretability and how we currently don’t understand AI and once it becomes superintelligent, it becomes super-unintelligible

It’s clearly written by somebody who understands ML and cares about AI safety and is really well written. Can find the whole webcomic here