B300 vs H100

The FP16 dense figure of 2250 TFLOPS on the B300 exceeds the corresponding 989 TFLOPS dense figure on the H100 by more than double. This difference translates to faster matrix operations during training and inference when both GPUs run dense FP16 workloads. The FP32 rating of 75 TFLOPS on the B300 also surpasses the 67 TFLOPS rating on the H100. Memory bandwidth of 8000 GB per s on the B300 permits larger batch sizes than the 3350 GB per s bandwidth on the H100 during memory intensive phases. When sparsity is enabled the B300 FP16 figure with sparsity reaches 4500 TFLOPS while the H100 reaches 1979 TFLOPS with sparsity. The same pattern holds for FP8 dense where the B300 records 4500 TFLOPS against 1979 TFLOPS on the H100.

The B300 suits workloads that require the 262 to 288 GB VRAM capacity and 8000 GB per s bandwidth. Large scale LLM training benefits from the FP16 dense rating of 2250 TFLOPS and the FP8 dense rating of 4500 TFLOPS. The 1400 W TDP is acceptable when maximum throughput per GPU takes priority over power draw.

Choose the H100 when the model fits in 80 to 94 GB and the extra throughput of the B300 would go unused, since it has the same NVLink and InfiniBand options for multi-GPU jobs. Choose it outright for FP64 HPC, where 34 TFLOPS against 1.25 TFLOPS is a 27.2 times gap. Its 1979 TOPS INT8 also far exceeds the 187.5 TOPS of the B300, but that matters only if your inference stack is locked to INT8 rather than FP8 or FP4, and the 700 W TDP is a concern for the operator rather than the renter.