Brief Review: If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All

Recently I pondered what 2 upcoming books would teach me, the first is done, but not dusted - anyone reading it should not move on too quickly...

There are already many reviews over on substack and goodreads of this book, so the most important contribution I can make is to urge people to educate themselves with a compelling book on the topic of ASI risks to humanity’s existence. The book:

• establishes the author’s credentials in the space (they are not mere armchair substack pundits, but multi-decade AI researchers).

• lay out the author’s arguments supported by credible sources and examples and addressing common objections.

• provides a dramatic scenario (a story) with a reasonable percentage likelihood of happening.

• The conclusion is to “back-off” the race to ASI.

• calls-to-action and who can help.

• provides an online resource for extended objection handling.

Key points:

1. ASI, (by prevailing definitions) is “more intelligent than all of humanity”. Humans are conditions to believe they can control their environment as they are the apex thinker. But ASI intelligence will be the new apex IQ. Think about the relationship with your pets, then reverse it (cats excluded).

2. Humans are good at restricting/preventing the reoccurrence of destructive events: by awareness, collaboration, agreement. One example given is the post world war 2 restriction of nuclear arms.

3. Human intelligence is very poor at detecting/stopping the first instance of a destructive event.

4. That ASI escaping control of its creators will only ever be a first instance. ASI’s apex IQ will have superior abilities to escape its lab sandbox, avoid detection, mitigate efforts to shut it down*.

5. ASI killing humanity is certain**. This prediction is easy. Predicting WHEN this will happen is very, very hard.

6. That heavily funded AI startups have a deeper set of biases against caution because they are in an “arms-race” to get to ASI. Their view is when they invent ASI they will experience wealth and fame and world domination. With the “leap-of-faith” that they can control it.

7. Politicians (both Rishi Sunak and Xi Jinping have publicly noted that risk of uncontrolled AI) and that policy makers can create global agreements similar to the nuclear arms treaties to monitor and commit to destroy AI lab facilities that breach the treaty.

8. That public officials not be concerned at looking alarmist of hysterical armed with information provided within this book.

9. That journalists write with awareness of the ASI issue.

10. That others readers may educate, influence and vote in their own circles to heighten awareness.

I hope taking a few minutes to write this brief review aids in the last point.

* Most people dismiss this scenario’s credibility. But it was only a few years ago it was asserted that we would never be so stupid to let AI’s on the internet. We also can’t conceive that an AI can run a 1000 thought experiments and assess results in the time it takes for a human finish typing this sentence.

** Parts of humanity may be kept alive for utilitarian purposes only as long as they can’t be replaced. That an ASI will be so rational that is won’t be sentimental about keeping us as pets. Community were essential for humanities genes to propagate and banding together created distributed cognition and brain power to become the apex predator in all environments. AI intelligence is alien in the sense that these needs are theoretical, not embodied.