RTX 5070 vs V100

The FP16 dense rating of 61.7 TFLOPS on the RTX 5070 falls below the 125 TFLOPS dense rating on the V100 so training workloads that rely on dense FP16 operations complete faster on the V100. The FP16 with sparsity rating of 123.5 TFLOPS on the RTX 5070 matches the dense rating of the V100 yet remains below any sparsity figure from the V100 since none is published. Memory bandwidth of 900 GB/s on the V100 exceeds the 672 GB/s on the RTX 5070 and therefore supports larger batch sizes during both training and inference before memory capacity limits are reached.

Choose the RTX 5070 for single card inference of models that fit in its 12 to 16 GB of GDDR7. It is the only one of the two with published FP8 (123.5 TFLOPS dense) and FP4 (493.9 TFLOPS dense) figures, so quantized serving of small models runs with modern kernels the V100 cannot use. It also draws 250W against 300W and sits on PCIe 5.0 rather than PCIe 3.0, which matters for host to device transfer. Do not choose it for anything that needs more than 16 GB or that must scale across several cards, since it lists PCIe only with no NVLink.

Choose the V100 for training and fine tuning runs that need its 32 GB HBM2 option, its 900 GB/s of bandwidth or its 125 TFLOPS of dense FP16, which is about 2.0x the 61.7 TFLOPS the RTX 5070 lists. Choose it as well when you need to link several cards over NVLink, or when you need FP64 at 7.8 TFLOPS, which the RTX 5070 does not publish. Keep in mind it is a Volta part with no published FP8 or FP4 figures, so it gains nothing from the low precision inference formats newer models are shipped in.