L40S vs RTX 3090

Dense FP16 throughput on the L40S reaches 362.05 TFLOPS while the RTX 3090 reaches 71 TFLOPS, producing a ratio of roughly five times higher compute on the L40S. The same ratio holds when sparsity is enabled, with 733 TFLOPS on the L40S versus 142 TFLOPS on the RTX 3090. FP32 performance follows a similar pattern at 91.6 TFLOPS versus 35.6 TFLOPS. These deltas translate into faster iteration during training loops and higher token throughput during inference when models fit within the available memory. Memory bandwidth of 864 GB/s on the L40S versus 936 GB/s on the RTX 3090 implies modestly smaller sustainable batch sizes on the L40S for bandwidth-bound kernels. The 48 GB VRAM capacity on the L40S compared with 24 GB on the RTX 3090 nevertheless permits larger models or bigger batches before swapping occurs. INT8 dense performance of 733 TOPS on the L40S versus 284 TOPS on the RTX 3090 further widens the gap for quantized inference workloads.

Training or inference jobs that exceed 24 GB per GPU therefore run without partitioning on the L40S while FP32 scientific kernels gain from the 91.6 TFLOPS rating.

Applications that require the 936 GB/s memory bandwidth or NVLink interconnect select the RTX 3090. Certain multi-GPU setups that already rely on NVLink bandwidth may achieve higher scaling efficiency despite lower per-GPU compute figures.