Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference

The key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference, enabling prefix reuse across requests. In multi-tenant deployments this cache is shared across tenants, creating a timing side channel: an adversarial tenant can reconstruct another tenant...