SemiAnalysis: AMD’s MI355X beats Nvidia’s B200 on performance per dollar in a Kimi K3 agent test

SemiAnalysis’s announcement

Chip research firm SemiAnalysis said on X on Oct. 31 that AMD’s Anush Elangovan and his team achieved up to 2x better performance per dollar than Nvidia’s B200 on Kimi K3 agentic inference. EE Times describes Elangovan as AMD’s vice president of AI software. SemiAnalysis credits part of the gain to software work, including tuning kernels in AITER, AMD’s library of AI operators for ROCm, and an effort called MonoKernel decode.

The second factor, according to the firm, is memory. An MI355X server has 2,304 GB of HBM, which lets it avoid crossing the slower 8x400 Gbit/s scale-out network between servers. A B200 server has less than 1.5 TB of HBM, so it has to cross that network. The configurations in the InferenceX data reflect this: MI355X runs with eight-way tensor parallelism (TP8), while B200 adds two-stage pipeline parallelism (PP2), splitting the model across two servers. SemiAnalysis thanked Elangovan, Peng Sun, Ramine, Emad and the vLLM team for the work.

SemiAnalysis@SemiAnalysis_

ALERT🚨🚨🚨: AMD's @AnushElangovan and his team have achieved up to 2x better performance per dollar than even B200 on Kimi K3 Agentic Inference! This is due to various software optimization efforts, including tuning AITER kernels and MonoKernel decode.

Along with MI355X having 2304GB of HBM per server, this allows it to avoid crossing slow 8x400Gbit/s scale-out networking between servers, which B200 needs to do due to having less than 1.5TB of HBM.

Great work to Anush, Peng Sun, Ramine, Emad, and the vLLM team for this amazing work!

How the test works

The measurement comes from InferenceX, SemiAnalysis’s continuous open-source benchmarking platform. The model is Kimi K3, Moonshot AI’s 2.8 trillion parameter open-weight model, running at FP4. The scenario is AgentX, introduced in August, which replays real coding agent sessions instead of fixed-length prompts. According to InferenceX, context grows turn over turn and most of each request is served from cache rather than recomputed. That makes KV cache transfer between nodes, prefix-aware routing and cache capacity matter alongside raw chip throughput.

On cost, SemiAnalysis uses its own total cost of ownership (TCO) model. Assuming ownership at large hyperscaler volume, it puts the cost per chip per hour at $1.73 for B200 and $1.50 for MI355X, based on its July 2026 pricing surveys and AI cloud TCO model. These are SemiAnalysis estimates, not rental or API prices.

The numbers

Cost per million tokens by tokens per second per user (Oct. 27 data)

• 57 tok/s: $0.089 on B200, $0.046 on MI355X
• 100 tok/s: $0.179 on B200, $0.077 on MI355X
• 144 tok/s: $0.275 on B200, $0.146 on MI355X

In SemiAnalysis’s data, the ratio of total tokens per dollar ranges from 1.74x to 2.45x depending on the operating point. The gap in hourly chip cost is much smaller than the gap in cost per token. Most of the difference therefore comes from how many tokens each system produces in the same time.

Limits

The result covers one model, one scenario and a comparison against a B200 server. On the same model, InferenceX shows Nvidia’s rack-scale GB300 NVL72 as cheaper than MI355X: $0.038 versus $0.059 per million tokens at 79 tok/s per user. MI355X does come out ahead of GB200 NVL72 ($0.050 versus $0.068 at 71 tok/s) and B300 ($0.048 versus $0.118 at 62 tok/s) on the same model.

Results also vary by software stack and model. In August, SemiAnalysis wrote that MI355X, running AMD’s own ATOM inference engine, beat GB300 NVL72 on part of the Kimi K3 curve. Another post published the same day found Nvidia up to 5x cheaper per token on GLM 5.3 at 150 tok/s per user. Figures on the live pages can change as new runs come in.