GB300 vs RTX 3090

The 2250 TFLOPS FP16 dense rating on the GB300 exceeds the 71 TFLOPS FP16 dense rating on the RTX 3090 by a factor of 31.7 while the 4500 TFLOPS FP16 with sparsity rating exceeds the 142 TFLOPS FP16 with sparsity rating by the same factor. This delta allows the GB300 to complete matrix operations in training and inference at substantially higher throughput when operating in dense mode or sparsity mode. The 75 TFLOPS FP32 rating on the GB300 likewise exceeds the 35.6 TFLOPS FP32 rating on the RTX 3090. Memory bandwidth of 8000 GB per second on the GB300 supports larger batch sizes than the 936 GB per second bandwidth on the RTX 3090 because data movement keeps pace with the higher arithmetic throughput. The 187.5 TOPS INT8 dense rating on the GB300 remains below the 284 TOPS INT8 dense rating on the RTX 3090 yet the GB300 INT8 with sparsity rating of 375 TOPS still exceeds the RTX 3090 INT8 with sparsity rating of 568 TOPS only when sparsity is applied.

The GB300 suits training or inference of models that require more than 24 GB VRAM because its 262 to 288 GB HBM3e capacity accommodates larger layers without partitioning.

The RTX 3090 suits inference or fine tuning tasks that fit within 24 GB VRAM and operate at INT8 dense throughput where its 284 TOPS rating exceeds the 187.5 TOPS rating of the GB300. Lower 350 watt TDP and PCIe form factor also align with deployments that cannot accommodate 1400 watt SXM6 modules.