BF16, FP8 & Low-Precision Numerics¶

Overview¶

Beyond fp16 lies the frontier: bf16 (fp32-range, low precision) and FP8 (e4m3/e5m2, ~8-bit) power modern LLM training and inference. The win is memory and speed; the cost is careful numeric engineering: scaling, loss scaling, and overflow watch.

bf16: 8-bit exponent (same range as fp32), 7-bit mantissa. Tolerates tiny gradients— no scaler.
e4m3 (FP8): 4-bit exponent, 3-bit mantissa, max ~448— great for activations/weights in inference, tiny range.
e5m2 (FP8): 5-bit exponent, 2-bit mantissa, max ~57344— larger range, coarser, good for gradients.
FP8 needs scaling (per-tensor or per-channel) to land values in its narrow range.

The #1 FP8 mistake: values outside [~6e-3, ~448] round to 0 or inf silently. Always scale before casting.

bf16— Why LLMs Use It¶

import torch

x = torch.tensor([1e-6, 1e-3, 1e5], dtype=torch.float32)
print("fp32:", x.tolist())
print("fp16:", x.to(torch.float16).tolist()) # 1e-6 -> 0 (underflow!)
print("bf16:", x.to(torch.bfloat16).tolist()) # range preserved, precision lost

# bf16 matmul — no scaler, same code as fp32:
with torch.amp.autocast("cuda", dtype=torch.bfloat16):
 y = model(x) # works with fp32-loss immediately

bf16 + fp32 master weights is the standard LLM recipe: 2x memory win, stable gradients, zero scaler drama.

-

FP8— The 8-Bit Frontier¶

import torch

# two fp8 dtypes
print(torch.finfo(torch.float8_e4m3fn)) # max ~448, 3-bit mantissa
print(torch.finfo(torch.float8_e5m2)) # max ~57344, 2-bit mantissa

x = torch.tensor([0.5, 1.0, 100.0], dtype=torch.float32)
print(x.to(torch.float8_e4m3fn)) # 100 is fine; >448 would clamp to inf

The scaling dance (per-tensor)¶

def to_fp8(x, dtype=torch.float8_e4m3fn):
 scale = torch.finfo(dtype).max / x.abs().max()
 return (x * scale).to(dtype), scale # scale to fit range

def from_fp8(qx, scale):
 return qx.to(torch.float32) / scale

On H100+ with tensor cores, FP8 GEMMs run 2x bf16 speed. Frameworks (torchao, DeepSpeed, NeMo) handle scaling internally— but you must still verify with your own calibration data.

Choosing the Right Low-Precision Path¶

Component	Typical dtype	Why
Weights (train)	fp32 master / bf16 copy	accumulate in fp32
Activations (train)	fp16/bf16 autocast	range decides fp16 vs bf16
Gradients (train)	fp32 (or e5m2 in FP8 training)	range matters most
Weights (inference)	int8/fp8/e4m3	memory-bound wins
KV cache (inference)	fp8/int8	huge memory win

-

Numeric Failure Modes & Guards¶

def guard_fp8(x, name="tensor"):
 finfo = torch.finfo(torch.float8_e4m3fn)
 ratio = x.abs().max() / finfo.max
 if ratio > 0.8:
 print(f"WARNING: {name} near FP8 max, scale needed: {ratio:.2f}")

# NaN/Inf detector for low-precision training
def check_finite(t, name):
 if not torch.isfinite(t).all():
 print(f"NON-FINITE in {name}"); return False
 return True

Loss Scaling in Low Precision (beyond fp16)¶

fp8 training often scales gradients per-layer or per-tensor; bf16 usually skips it. General pattern:

class DynamicGradScale:
 """Very simplified per-step scale estimator."""
 def __init__(self, init=1.0):
 self.s = init
 def update(self, grad_norm):
 if not torch.isfinite(grad_norm):
 self.s *= 0.5
 elif grad_norm < 1e-4:
 self.s *= 2.0
 return self.s

For real FP8 training use proven libraries (torchao, NVIDIA transformer-engine)— hand-rolled scaling is a research project.

Practical Decision Rules¶

bf16 everywhere for training unless your hardware lacks fast bf16 (then fp16+scaler).
FP8 for inference & frontier training only with a scaling-aware library.
Never cast to fp8 without scaling, and clamp per-tensor.
Keep fp32 for: master weights, loss, embeddings in huge-vocab models, and any reduction.
Validate with a tiny subset before committing a whole run.

def safe_bf16_fp32(model):
 # common pattern: bf16 activations, fp32 master weights handled by optimizer
 return model.to(torch.bfloat16) # + separate fp32 optimizer copy in frameworks

-

Key Takeaways¶

bf16 = fp32 range, lower precision— safe default for training.
FP8 = 8 bits with big range constraints— scaling is mandatory, use libraries.
e4m3 for weights/activations, e5m2 when range is the concern (gradients).
Master weights and reductions stay fp32; only compute-heavy ops drop precision.
Always monitor non-finite values and validate accuracy vs fp32 baseline.

-