Google open-sources TPU Sync library

Google Cloud has open-sourced TPU Sync, a native C++ library designed to move KV cache blocks and model weights between TPU high-bandwidth memory, host DRAM, storage, and data center networks. The project targets accelerator bottlenecks in long-context disaggregated LLM serving and reinforcement learning training loops. Google reports substantial latency reductions and time-to-first-token improvements for large language models using the library. Open article →