FOSCAR: Accelerating Decode Attention over 2- and 4-Bit KV Caches through Affine Folding
FOSCAR: Accelerating Decode Attention over 2- and 4-Bit KV Caches through Affine Folding. It centres on Benchmarks, and also names GitHub. Reported by arXiv. Bharat Hunt files it under AI Hardware — the section covering chips, accelerators, data centres, on-device inference and supply.
Written by Bharat Hunt from the headline and the coverage below. The original reporting is the source of truth.