FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper
FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper. It centres on Llama, and also names Inference and NVIDIA. Reported by arXiv. Bharat Hunt files it under AI Models and AI Hardware — the section covering a new or updated model, its capabilities, benchmarks or availability.