It regresses compared to o3 on a harder question set.

GPT-5 doesn't show improvement on GeoGuessr.

Grok 4 does not compete well in GeoGuessr, scoring below the average player.

New Gemini 2.5 regresses slightly for GeoGuessr.

The new batch of models frequently guessed on the wrong continent. Rounds like this are epitomic of elementary mistakes Opus made (road lines). The full response is available here:

Claude takes a heavy hit on geolocation in its new release: Opus 4 with thinking barely ekes out even Claude 3.5 Haiku, with Sonnet 4 performing worse. This is a 38% decrease in performance (per average score).

We note unusual behavior of the new model: the new Gemini 2.5 improves on the old version by 100 points; however, with search, it hallucinates much more often and gets a worse overall score. You can read about why that might be here:

The new Gemini 2.5 is the first to get two perfect scores in 100 rounds on GeoBench! Impressively, both without search tools.

Human baselines have been added👀

New o3-high results on real-life photos — it performs measurably worse than Gemini 2.5 Pro, with a 10% refusal rate.

Thanks for featuring us @AIExplainedYT :)

New site, with results for Gemini 2.5 Pro with search tools and o3 high!