Export, Deployment & Production¶
Overview¶
Training is the fun part; deployment is where models die. This chapter covers turning an eager PyTorch model into something fast, small, and callable outside Python: TorchScript, ONNX, torch.export, served inference, and C++/libtorch.
Topics¶
- Export — TorchScript, ONNX, torch.export - The three exit doors, when to use each, and their gotchas.
- Inference Optimization & Serving - Batching, caching, async I/O, TorchServe/Triton-style patterns.
- C++ & libtorch Integration - Loading exported models in C++ apps without Python.
Key Patterns¶
- Export is a contract: pin input shapes, dtypes, and control flow before exporting.
- Prefer
torch.export(new) over TorchScript (legacy) for new projects; ONNX for interop. - Quantize + compile before exporting to lock in deployment wins.
- Serve with stateless handlers + batching; keep Python out of the hot loop when possible.