Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM

Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM. It centres on Inference, and also names vLLM and Benchmarks. Reported by AWS. Bharat Hunt files it under AI Models — the section covering a new or updated model, its capabilities, benchmarks or availability.

Written by Bharat Hunt from the headline and the coverage below. The original reporting is the source of truth.