Google’s new Gemini Pro model has record benchmark scores — again

Benchmarks are a trap. Google's already best-in-class on evals. Clients care about (1) cost per token, (2) latency, (3) reliability, (4) whether it works for their specific thing. Gemini 3.1 being "better" on benchmarks means almost nothing unless it's also cheaper or faster.

The real story's that Google's throwing everything at LLM performance while pricing stays confusing. They're winning on paper, losing in the sales call.