L40 vs RTX 4090

Dense FP16 performance stands at 181.05 TFLOPS on the L40 and 165.2 TFLOPS on the RTX 4090. The FP16 to FP32 ratio equals 2 on both cards because 181.05 divided by 90.5 yields 2 and 165.2 divided by 82.6 yields 2. This ratio indicates that mixed precision training or inference can double throughput relative to pure FP32 execution on either card. Sparse FP16 performance reaches 362.1 TFLOPS on the L40 and 330.4 TFLOPS on the RTX 4090. Memory bandwidth of 864 GB per second on the L40 versus 1008 GB per second on the RTX 4090 determines the maximum sustainable batch size when activation tensors exceed on chip cache capacity. Higher bandwidth on the RTX 4090 supports larger batches in bandwidth bound kernels while the L40 larger 48 GB capacity supports models whose weights alone exceed 24 GB.

The L40 suits workloads that require 48 GB of memory such as large language model inference with batch sizes that fit only in the larger frame buffer. Its 300 W thermal design power also fits installations where power delivery or cooling capacity remains constrained to 300 W per card. Dense FP8 performance of 362 TFLOPS further benefits quantized inference pipelines that stay within the 48 GB limit.

Its INT8 dense performance of 660.6 TOPS exceeds the L40 figure of 362 TOPS and therefore accelerates integer quantized inference when memory capacity of 24 GB proves sufficient.