SWE-Game: even the best model scores below 60 out of 100 on building games in Godot
Eleven researchers from Shanghai Jiao Tong University and five other institutions have released SWE-Game, a benchmark of how well AI coding agents develop games, posted on arXiv on September 27 and revised on October 7. Its 247 tasks, built on 41 working reference games made in Godot, include making a game from a brief or a design document, completing skeleton code, fixing injected bugs and porting a game to Unity. Scores out of 100 combine in-engine checks with a visual rubric. Of six models, Anthropic’s Opus 5 led all five task types, yet its best score on the three game-building tasks was 59.68, and 50.38 when working from a brief. On bug repair, it scored 83.46. Each model ran in its own agent framework.
Sources
- arXiv, “SWE-Game: Can Coding Agents Build the Games We Want?”, Paper (v1 Sept 27, v3 Oct 7), (arxiv.org)
- arXiv, “SWE-Game: Can Coding Agents Build the Games We Want?”, Table 3: scores and scoring method, (arxiv.org)
- Hugging Face, “Paper page - SWE-Game: Can Coding Agents Build the Games We Want?”, Daily Papers page, (huggingface.co)
- Wikimedia Commons, “File:Godot logo.svg”, Godot logo: Andrea Calabró, CC BY 4.0, (commons.wikimedia.org)
About this story
This story was posted on Instagram by @jarrus.tech on Oct. 13, 2026.
Spotted an error in this story? [email protected] · Instagram
Short link: thejarrus.com/en/swe-game
This story in Turkish: SWE-Game: Godot’ta oyun inşa görevlerinde en iyi model bile 100 üzerinden 60’ı geçemedi
On the same topic
The US is again letting AI companies share their models with Britain’s AI Security Institute before release
According to The Telegraph, the US dropped its request that AI companies hold new models back from Britain’s AI Security Institute.
CNBC: Small groups like METR, Apollo Research and Transluce are emerging as AI’s safety gatekeepers
AI labs are leaning on independent evaluators, CNBC reported, but there are still no standards for funding, access or reporting.
Only 3.6% of Chinese AI model releases came with a published safety test result
Only 31 of 857 model releases from nine major Chinese developers came with a published safety evaluation result, SemiAnalysis found.