Flide: Accelerating Decision Inference with Exact State Forking. It centres on Inference, and also names vLLM. Reported by arXiv. Bharat Hunt files it under AI Hardware — the section covering chips, accelerators, data centres, on-device inference and supply.
Written by Bharat Hunt from the headline and the coverage below. The original reporting is the source of truth.
Measuring the Serving Stack Instead of the Model: Hidden Confounds in Local Tool-Use Evaluation. It centres on Benchmarks, and also names Ollama and Phi. Reported by arXiv. Bharat Hunt files it under AI Tools and AI Models — the section covering applications people use — assistants, editors, search, productivity.
Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM. It centres on Inference, and also names vLLM and Benchmarks. Reported by AWS. Bharat Hunt files it under AI Models — the section covering a new or updated model, its capabilities, benchmarks or availability.
Improve TTFT for multi-turn agentic workloads with bi-directional KV transfer in vLLM. It centres on vLLM, and also names Amazon. Reported by AWS. Bharat Hunt files it under AI Agents — the section covering autonomous and tool-using systems, agent frameworks, MCP and orchestration.
Generate images and video with vLLM-Omni on SageMaker AI – Part 2. It centres on vLLM, and also names Amazon and Flux. Reported by AWS. Bharat Hunt files it under AI Models — the section covering a new or updated model, its capabilities, benchmarks or availability.