AMD reports more than double the throughput in MI355X long-prompt test
The benchmark put most of the work before the reply. Its results are most relevant to AI services that repeatedly process the same source material.
Advanced Micro Devices (Nasdaq: AMD) said Wednesday that software changes lifted total throughput on eight MI355X AI accelerators to 2.45 times the baseline rate. AMD ran four requests at once through the same GLM-5.2 model and hardware in each setup. Throughput rose from 162,559 to 397,922 tokens a second. Each request supplied a 64K-token prompt and asked for a single token back, putting most of the work in reading the input. AMD’s November 11 benchmark
AMD’s higher figure of 628,866 tokens a second came with 50 requests running at once. The baseline peaked at four. Comparing the two peaks folds a change in the number of simultaneous requests into the throughput comparison.
What the chips could reuse
Before an AI model answers, it reads the prompt and stores information it can use later. AMD split that work across the eight chips and changed how the software stored and read that information. In the 64K and 128K tests, the first 90% of each prompt was shared across requests, so the software could reuse much of that work. Think of a service answering different questions about the same long report. The benchmark measures AMD’s software changes together. AMD’s methods and test conditions
In a separate test with a million-token input and one request at a time, AMD reported that the average wait for the first token fell to 6.34 seconds from 45.66 seconds. That measures when a reply begins. For a cloud operator, the unanswered question is how many complete requests those eight chips can finish within a fixed response-time target when prompts vary and answers run longer.