FP8 E4M3 vs E5M2

Same byte, different split. E4M3 gives one exponent bit to the mantissa: twice as fine (8 steps per power of two vs 4), over 18 binades of range instead of 32. For a KV cache take E4M3. Memory is identical, so precision decides.

At a glance

Limits: Micikevicius et al. 2022, Table 1 (the encoding OCP OFP8 standardised). Code counts: all 256 codes enumerated with ml_dtypes 0.6.0. Rounding error as a share of the power of two below the value.

Exponent bits buy range, mantissa bits buy precision

A normal FP8 value is (−1)^S × 1.M × 2^{E − bias}; an all-zero exponent gives subnormals, 0.M × 2^{1 − bias}. After the sign there are 7 bits to share.

• Exponent bits set how many powers of two are reachable. 5 bits reach 32 binades, 4 bits 18.
• Mantissa bits cut each power of two into 2^M equal steps. 3 bits: 8 steps. 2 bits: 4. The step scales with the power of two, so the worst-case error in percent is the same in every binade above the smallest normal.
• E4M3 drops infinity. IEEE style would reserve the whole top exponent for inf/NaN and stop at 240. The “fn” encoding (finite + NaN) keeps one NaN pattern per sign and turns the rest into 256, 288 … 448: one extra binade (paper §3.1).
• E5M2 is IEEE style. Same exponent width and bias as FP16, so it is FP16 with 8 fewer mantissa bits and nearly the same reach: 57,344 vs 65,504.

Worked example

ml_dtypes 0.6.0 in python:3.12-slim, round to nearest even, no scale. Its cast does not saturate, so 1000 becomes NaN rather than 448.

3.3 = 1.65 × 2¹. E4M3 has mantissas 1.625 and 1.75 either side, and 1.625 is nearer: 3.25. E5M2 has 1.5 and 1.75, and 1.75 is nearer: 3.5.

0.001 is below E4M3’s smallest normal (2⁻⁶) and nearest its smallest subnormal (2⁻⁹), so it nearly doubles. For comparison, BF16 stores 3.3 as 3.296875 (−0.09 %) and 0.1 as 0.10009765625 (+0.10 %).

The scale moves the window, it does not widen it

E4M3 gets its range back with a scale s: store x ÷ s, read back q × s. One FP32 scale per tensor, or per head, is a few bytes per layer.

Same ml_dtypes run. 0.001 ÷ 4 = 0.00025, below half of E4M3's smallest subnormal: flushed to zero.

0.001 to 1000 is 19.9 binades. E4M3 covers 17.8 (2⁻⁹ to 448), so one scale keeps one end. E5M2 covers 31.8 and keeps both, at twice the rounding error everywhere.

How the engines set it:

• vLLM: one scale per K and V tensor, or per KV head with an llm-compressor calibration (Flash Attention backend only). Without calibration “all quantization scales are set to 1.0”.
• SGLang: “only per-tensor (scalar) scaling factors”. They come from the checkpoint (ModelOpt k_scale / v_scale) or --quantization-param-path. Missing, they default to 1.0, “which may cause accuracy issues”.

At scale 1.0, any K or V value past ±448 is out of E4M3’s range.

Which one for a KV cache

• E4M3. It has twice the precision, and the engines default to it. vLLM: “CUDA 11.8+ supports fp8 (=fp8_e4m3) and fp8_e5m2. ROCm (AMD GPU) supports fp8 (=fp8_e4m3).” SGLang: “Use fp8_e4m3 for better accuracy (recommended), fp8_e5m2 for larger dynamic range”.
• Memory does not decide. Both are 1 byte: half of BF16, so ~2x the tokens in the same KV pool and half the KV bytes read per decoded token, the same for both.
• E5M2 if no scale can be had and K/V may pass 448. Not on ROCm under vLLM.
• ±240 in some docs. SGLang’s KV page gives E4M3’s range as ±240. That is the max of the FNUZ variant (float8_e4m3fnuz, bias 8), native on AMD MI300; MI350 and NVIDIA use OCP E4M3, max 448. Read the variant before trusting a range.

Measured here: E4M3 KV on 2x DGX Spark

What we saw on our hardware, not a general law. Qwen3.8 Flash-Next on vLLM, tensor parallel over two GB10 boxes. Its KV default moved from BF16 to fp8_e4m3 on 2026-10-26. fp8_e5m2 has never run here.

The gateway's per-minute perf log, 2026-10-26, production traffic at ~44–79K tokens of context per request. Same concurrency, not a controlled benchmark.

Likely reason, not proven: this model’s attention backend (vLLM’s QWEN4_EXP_QSA_TRITON) dequantises FP8 to BF16 in registers on every read. That adds work instead of saving bandwidth. On this kernel FP8 KV bought ~2x context capacity, not speed.

The 4-bit side of the same question: KV cache: FP8 vs NVFP4 and FP4 KV cache on the local fleet. Every kv_cache_dtype and who refuses it: KV type. E4M3 is also NVFP4’s block-scale format: MXFP4 vs NVFP4. The 16-bit baseline both are weighed against: BF16 vs FP16.

Sources

• FP8 Formats for Deep Learning — Micikevicius et al. (NVIDIA, Arm, Intel), arXiv 2209.05433
• OCP 8-bit Floating Point Specification (OFP8) rev 1.0 — Open Compute Project, 2023
• Using FP8 with Transformer Engine — NVIDIA
• Quantized KV cache — vLLM docs
• vllm serve CLI, --kv-cache-dtype — vLLM docs
• Quantized KV cache — SGLang docs
• FP8 numbers — AMD HIP docs
• MI300 / MI350 workload optimization — AMD ROCm docs
• ml_dtypes — JAX/ML numeric types for NumPy

ai llm engineering