# Artificial Analysis rebuilt its AI ranking around private tests: Intelligence Index v5

> 60% of the index now relies on private test questions. The gap between open and closed models grew from 3 points to 12.

- Publisher: JARRUS (@jarrus.tech), thejarrus.com
- Story date: Oct. 27, 2026 (2026-10-27)
- Published: 2026-10-29
- Tags: Models, Research, Pricing and access
- URL: https://thejarrus.com/en/news/intelligence-index-v5/

### The index was rebuilt

Independent evaluator Artificial Analysis released version 5 of its Intelligence Index, the most widely cited AI ranking, on Oct. 27. The company calls it the biggest upgrade since version 4 launched in January. For the first time, a majority of the index, **60%**, relies on private test sets or held-out answers. The same day it launched Progress Index v1, which places language models released since 2022 on a single capability scale.

https://x.com/ArtificialAnlys/status/2115265133467635877

### The launch ranking and Gemini 4 Carbon

At launch, Claude Opus 5.5 led with 58, followed by GPT-6 Astra at 57, GPT-6.1 Sol at 55, Claude Sonnet 5.5 at 54 and GPT-6.1 Terra at 51; Anthropic and OpenAI held the top five. Gemini 4 Argon scored 50. A day later, Google's newly announced Gemini 4 Carbon took first place **with 59**.

![Artificial Analysis Intelligence Index v5 ranking: Claude Opus 5.5 58, GPT-6 Astra 57, GPT-6.1 Sol 55, Claude Sonnet 5.5 54, GPT-6.1 Terra 51, Gemini 4 Argon 50](https://thejarrus.com/gorsel/c60-intelligence-index-v5.webp)
*Chart: [Artificial Analysis, Intelligence Index v5](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5) (Oct. 27, 2026). Ranking at launch; Gemini 4 Carbon (59) was added later.*

### Four equal categories, nine tests

The index has four categories, each weighted at 25%. Knowledge work covers AA-Briefcase (15%), AutomationBench-AA (5%) and GDP.pdf (5%). Coding has FrontierCode 2.0 (15%) and Terminal-Bench 4.0 (10%), and science has Terminal-Bench-Science (15%) and the hard subset of CritPt v2 (10%). The general category includes AA-Omniscience v1.1 (15%) and HLE-Diamond (10%).

Two tests are new. **FrontierCode 2.0**, built by Cognition, grades an agent's code change on whether a repository maintainer would merge it; its tasks are kept private to avoid contamination. Terminal-Bench-Science has 70 agentic scientific research tasks run in a terminal. A 1,000-question HLE-Diamond, split evenly between reasoning and knowledge questions, replaced Humanity's Last Exam. AA-Omniscience dropped saturated and public questions, and a private 40-question hard subset of CritPt v2 is run with a code-execution tool. GDPval-AA v2.1, SciCode and AA-LCR v1.1 were removed.

The leaders differ by test. Claude Opus 5.5 leads FrontierCode 2.0 at 46% and HLE-Diamond at 68%. GPT-6 Astra leads Terminal-Bench-Science at 63% and CritPt v2 at 72%. Kimi K3.1 Pro tops Terminal-Bench 4.0 at 67%, and Gemini 4 Argon leads AutomationBench-AA at 78%.

### Open models fell further behind

Open-weights models lost ground in the new version. The gap between the best open model and the leader was 3 points in the previous version and is now **12 points**. The top open model, Kimi K3.1 Pro, scores 46, but its weights haven't been released yet. Among models with released weights, DeepSeek V4.1 Flash 1017 and GLM-5.3 lead at 32, 26 points behind the leader and lower than Claude Haiku 5.5 (36).

![Artificial Analysis Intelligence Index v5 by open weights and proprietary: proprietary models in black, open-weights models in blue](https://thejarrus.com/gorsel/c60-acik-kapali-modeller.webp)
*Chart: [Artificial Analysis](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5) (Oct. 27, 2026). Black: proprietary, blue: open-weights models.*

### Cost varies more than 13x

Among the 10 highest-scoring models, the cost per index task varies by more than 13x. GPT-6.1 Terra is the cheapest at $0.96 and Claude Fable 5.1 the most expensive at $13.04. Claude Opus 5.5 costs $10.04 at max effort and $0.98 at low effort, GPT-6 Astra $6.24 and GPT-6.1 Sol $1.27. The median number of output tokens per task also rose from 35,000 in version 4.3 to 59,000.

![Artificial Analysis Intelligence Index v5 versus cost per task, log scale](https://thejarrus.com/gorsel/c60-maliyet.webp)
*Chart: [Artificial Analysis](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5) (Oct. 27, 2026). Horizontal axis: cost per task (USD, log scale).*

### The critique: private tests and trust

Researcher Nathan Lambert called the index "incredibly useful for the community and massively flawed!" in a post on Interconnects. He says private questions make it harder to train to the test, but they also stop the community from finding errors in the tests, so users have to put complete trust in the private benchmark builder. Lambert also notes that Artificial Analysis's business model is to get paid by labs to verify their results, which he treats as a trust issue. He argues the field needs several indices with different biases rather than a single intelligence index.

## Sources

1. [Artificial Analysis: Artificial Analysis Intelligence Index v5, Oct. 27, 2026](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5)
2. [Artificial Analysis (X): Announcement post and summary of results, Oct. 28, 2026](https://x.com/ArtificialAnlys/status/2115265133467635877)
3. [Artificial Analysis: Gemini 4 Carbon (High), Oct. 28, 2026](https://artificialanalysis.ai/models/gemini-4-carbon)
4. [Interconnects (Nathan Lambert): Artificial Analysis v5 and the benchmarker’s dilemma, Oct. 28, 2026](https://www.interconnects.ai/p/artificial-analysis-v5-and-the-benchmarkers)

Report an error: corrections@thejarrus.com