# “A planet-sized neural network would fail”

> But a neural network the size of a planet, given the same examples, would have failed in the same way.

Canonical URL: https://foom.hyperplex.org/p/planet

Chapter: Overcoming Bias / LessWrong / Rationality community

Category: neural nets & capabilities

Predicted: 2006

Resolution date: 2023

Status: falsified by events

Status detail: The prediction was resolved as falsified by events in 2023

Original source label: SL4, Jun 2006; contra Bill Hibbard

## Resolution

Planet-scale compute plus local search is precisely what produced GPT-4. Scale was the answer he ruled out.

## Context and outcome

Arguing against Hibbard's proposals, he held that supervised learning with local search had a general, unfixable failure mode that no amount of scale would touch.

The systems that later surprised him are supervised/self-supervised learning with local search at unprecedented scale, the exact configuration the planet-sized-network claim ruled out.

## Supporting receipts

> It is a general problem. … It is a problem that people run into, in general, with supervised learning using local search techniques for traversing the hypothesis space.

Context: Defending his smiley-face critique of Bill Hibbard as a general flaw of supervised learning, not one failed network.

Source: [SL4, 6 Jun 2006](http://sl4.org/archive/0606/15161.html)

> But you could have used a galactic-size neural network in the army tank classifer and gotten exactly the same result…

Context: Claiming the army tank-classifier failure was scale-independent; the spelling is the original's.

Source: [SL4, 6 Jun 2006](http://sl4.org/archive/0606/15161.html)

> But a neural network the size of a planet, given the same examples, would have failed in the same way.

Context: Telling Hibbard that no amount of scale fixes misclassified training targets.

Source: [SL4, 12 Jun 2006](http://sl4.org/archive/0606/15268.html)

## Sources

- [vs. Hibbard (2006)](http://sl4.org/archive/0606/15161.html)
- [planet-sized network](http://sl4.org/archive/0606/15268.html)