Chapter 03 · 2000–~2012
Singularity Institute for Artificial Intelligence
From 'we will build Friendly AI' to early catastrophic risk warnings
Overview
Yudkowsky’s first organization, founded in July 2000 with seed funding from Brian and Sabine Atkins. Its mission was to create a Friendly, self-improving Artificial Intelligence and thereby bring about the Singularity.
As the Institute's own technical efforts looked increasingly fruitless while a neural network renaissance bloomed, the Institute turned its attention toward the catastrophic risks posed by unaligned AI, which soon became central to its public work.
Predictions in this chapter
Neural nets: “certainly Not Sufficient”
2000 → 2023; falsified by events.
Hard takeoff in a basement within ten years
2001 → 2011; falsified by events.
“Three hours to transhumanity”
2001 → 2008; walked back.
Time to Singularity: “2005–2020, peak 2008–2010”
2001 → 2020; falsified by events.
SIAI ships transhuman AI ~2008–2010
2001 → 2010; falsified by events.
“Dead certain”: days to nanotech
2002 → 2023; falsified by events.