Prediction · economics & deployment
Codex: stagnation for 5 years or a massive leap in 2
Predicted in 2021 from takeoff debate with Paul Christiano, Nov 2021
“I feel like I would indeed not have the right to feel very surprised if Codex technology stagnated for the next 5 years, nor if it took a massive leap in 2 years and got ubiquitously adopted by lots of programmers.”
Resolution
hedged: couldn't lose. The prediction was assessed as hedged: couldn't lose in 2023.
Reality took the massive-leap branch, on the 2-year clock.
But a claim licensing both branches can never lose, so it doesn't win.
Timeline
- Prediction made
- 2021
- Assessment date
- 2023
- Chapter
- Chapter 05: MIRI: Research
- Verdict
- hedged: couldn't lose
Context and outcome
Christiano expected spending on Codex to rise quickly to something under $10 million a year, then double annually. Yudkowsky said he would be unsurprised by either five years of little progress or a dramatic leap to widespread use within two years. In other words, both stagnation and rapid growth were treated as compatible with the model.
Events followed the second path more closely. By early 2023, GPT-4 could handle substantial coding tasks, and by 2024 coding assistants were in widespread use, growing far faster than Christiano's simple doubling curve implied. I would still score the prediction as yellow rather than green, as Yudkowsky did identify relatively unregulated products such as Codex or, in his phrase, “anime waifus,” as likely places for a breakthrough. But the larger claim was broad enough to accommodate opposite outcomes, which was essentially Christiano’s objection to the model.
Supporting receipts
“well, if I was betting on something taking a big jump in income, I sure would bet on something in a relatively unregulated industry like Codex or anime waifus.”
The adjacent claim did stick its neck out by naming the breakout niche, while declining to actually bet.