Large Language Models: A New Moore’s Law?

Since GPT-3, LLM capability-per-dollar has roughly doubled every 8–12 months. GPT-4o now costs 1/20th what GPT-4 cost at launch for equivalent token throughput. No other infrastructure category has deflated this fast.

Teams designing RAG pipelines today are hardcoding cost assumptions that will be wrong within two quarters. Locking every agent call to Claude Opus because it topped your 2024 evals is just burning budget. Claude Haiku or GPT-4o mini handles 80% of classification and extraction tasks at 50x lower cost.