L40 vs RTX 3070

The FP16 dense performance of 181.05 TFLOPS on the L40 exceeds the 40.6 TFLOPS dense FP16 figure on the RTX 3070 and this gap translates to faster execution of training and inference steps that rely on dense FP16 arithmetic. The corresponding FP32 values of 90.5 TFLOPS and 20.3 TFLOPS indicate similar scaling for workloads that require full single precision. When sparsity is enabled the L40 supplies 362.1 TFLOPS FP16 with sparsity against 81.3 TFLOPS FP16 with sparsity on the RTX 3070. Memory bandwidth of 864 GB/s on the L40 versus 448 GB/s on the RTX 3070 permits larger batch sizes before memory capacity limits are reached. The 48 GB GDDR6 on the L40 further supports these larger batches compared with the 8 GB GDDR6 limit on the RTX 3070. All tensor comparisons remain within the same category of dense or with sparsity throughout.

The L40 serves workloads that require 48 GB GDDR6 memory and FP16 dense throughput of 181.05 TFLOPS. Its FP8 dense rating of 362 TFLOPS also suits inference pipelines that benefit from the higher memory bandwidth of 864 GB/s.

The RTX 3070 serves workloads constrained to 220W TDP and that fit inside 8 GB GDDR6 memory. Its INT8 dense rating of 162.6 TOPS remains sufficient for lighter inference steps that do not exceed the 448 GB/s bandwidth limit.