Google Open-Sources TPU Sync for KV Cache and Weight Transfer on TPUs

Google Cloud has open-sourced TPU Sync, a native C++ library that moves KV cache blocks and model weights between TPU HBM, host DRAM, shared storage, and the data center network. By Daniel Dominguez

Preview courtesy of InfoQ. The full article opens on their site.