Fordel StudiosComing soon: 10 Things That Matter in AI Right Now
What Happened
Each year we compile our 10 Breakthrough Technologies list, featuring our educated predictions for which technologies will have the biggest impact on how we live and work. This year, however, we had a dilemma. While our final picks encompass all our core coverage areas (energy, AI, and biotech, plus
Our Take
MIT Technology Review is replacing its annual '10 Breakthrough Technologies' list with '10 Things That Matter in AI Right Now,' citing the pace of AI change as incompatible with a once-a-year format.
For teams running evals, the timing problem is real: quarterly cycles against static benchmarks are already stale. GPT-4 to GPT-4o, Claude 3 to Claude 3.5 — capability gaps that break production assumptions now close in months. Static benchmark suites don't catch output distribution shifts before they hit users.
What To Do
Run evals against your production traffic monthly instead of quarterly because model updates — GPT-4o, Claude 3.5 Sonnet — change output distributions fast enough to invalidate assumptions between standard release cycles.
Cited By
React
Get the weekly AI digest
The stories that matter, with a builder's perspective. Every Thursday.