Google’s TPU Sync cut AI response waits in tests on Ironwood
Google reported shorter waits for AI responses in TPU Sync tests on seventh-generation Ironwood. Its new 8tx and 8i Max systems were announced separately.
Google published results Monday showing that its TPU Sync software shortened the wait for the start of an AI response for two model sizes in internal testing. The tests ran on seventh-generation Ironwood accelerators in July. Google announced two new eighth-generation systems the same day, the 8tx and 8i Max. The performance figures in the software report describe the older hardware. Google’s TPU Sync report · Google’s hardware announcement
Moving a model’s working memory takes time
Some AI systems use one set of accelerators to process a prompt and another to generate the answer. The second set needs the intermediate data from the first, known as the key-value cache. TPU Sync lets the system move that data while computation continues. The opportunity is to spend less accelerator time waiting for a handoff. TrendyVest’s guide to the AI memory bottleneck
Google reported that the median time to transfer the cache for Qwen3 235B fell from several seconds to about 230 milliseconds. It separately reported that the average wait for the first token, the first piece of a response, fell 60% with Qwen3 235B and 90% with a 480-billion-parameter model when few requests were running at once. Both results were measured against running without TPU Sync. The transfer figure measures a single step. The first-token figures measure the wait through the start of generation. Benchmark results and test conditions
What the new systems still have to show
Google says the 8tx uses the same silicon as the 8t and is designed for confidential computing. The 8i Max pairs two 8i chips to handle longer input contexts. Those are separate hardware changes. The Ironwood test gives customers a reason to test the software on systems they already use, with their own prompts, traffic and network configuration. Google’s description of the new systems
A shorter wait before the first token can improve the user experience even if the rest of the response arrives at the same speed. For buyers comparing cloud costs, the more useful question is how many requests a rented system can finish at a given response-time target. The published tests do not give that answer for the 8tx or 8i Max.