Chapter 04 · 2006–2012
Overcoming Bias / LessWrong / Rationality community
Debugging humanity, one very long blog post at a time
Overview
Robin Hanson founded Overcoming Bias, where Yudkowsky soon became the most prominent contributor. Across the Sequences, lessons about cognitive bias and epistemology accumulated into a shared intellectual web, a course of study, and then the basis of a community. LessWrong launched in 2009, carrying that project into a site of its own.
The earlier emphasis on accelerating AGI increasingly gave way to “raising the sanity waterline.” The hope was that a small group trained to reason unusually well might be able to build AGI without causing an existential catastrophe. The 2008 Hanson–Yudkowsky “Foom” debate put that hope beside a more basic disagreement over how AI progress would actually unfold, although the test itself was just two theorists arguing about a future that neither could see.
Predictions in this chapter
“A planet-sized neural network would fail”
2006 → 2023; falsified by events.
AGI = ten insights, a small team, one desktop
2008 → 2023; falsified by events.