Prediction · neural nets & capabilities
“A planet-sized neural network would fail”
Predicted in 2006 from SL4, Jun 2006; contra Bill Hibbard
“But a neural network the size of a planet, given the same examples, would have failed in the same way.”
Resolution
falsified by events. The prediction was resolved as falsified by events in 2023.
Planet-scale compute plus local search is precisely
what produced GPT-4. Scale was the answer he ruled out.
Timeline
- Prediction made
- 2006
- Resolution date
- 2023
- Chapter
- Chapter 04: Overcoming Bias / LessWrong / Rationality community
- Verdict
- falsified by events
Context and outcome
Arguing against Hibbard's proposals, he held that supervised learning with local search had a general, unfixable failure mode that no amount of scale would touch.
The systems that later surprised him are supervised/self-supervised learning with local search at unprecedented scale, the exact configuration the planet-sized-network claim ruled out.
Supporting receipts
“It is a general problem. … It is a problem that people run into, in general, with supervised learning using local search techniques for traversing the hypothesis space.”
Defending his smiley-face critique of Bill Hibbard as a general flaw of supervised learning, not one failed network.
“But you could have used a galactic-size neural network in the army tank classifer and gotten exactly the same result…”
Claiming the army tank-classifier failure was scale-independent; the spelling is the original's.
“But a neural network the size of a planet, given the same examples, would have failed in the same way.”
Telling Hibbard that no amount of scale fixes misclassified training targets.