NVIDIA H200
141 GB HBM3e $3.50 net / GPU-hour
The memory-bandwidth machine. 4.8 TB/s to 141 GB of HBM3e.
- Accelerator
- NVIDIA H200 Tensor Core GPU
- Memory
- 141 GB HBM3e
- Bandwidth
- 4.8 TB/s
- Tensor cores
- 4th-gen, FP8 Transformer Engine
- Host
- Dedicated server, single tenant
- Storage
- Local NVMe scratch
- Draw
- ~700 W under load
- Training and fine-tuning runs that need the whole GPU
- Throughput inference at large batch
- Memory-bound work — long context, big activations
- Large-dataset analysis and rendering