L40S vs RTX 5090

The FP16 to FP32 ratio on the L40S equals 362.05 TFLOPS dense FP16 divided by 91.6 TFLOPS FP32 while the same ratio on the RTX 5090 equals 209.5 TFLOPS dense FP16 divided by 104.8 TFLOPS FP32. This ratio indicates that the L40S supplies greater acceleration for mixed precision training and inference steps that rely on dense FP16 operations compared with the RTX 5090. Memory bandwidth of 1792 GB/s on the RTX 5090 exceeds the 864 GB/s figure on the L40S by a factor of roughly two and therefore supports larger batch sizes during inference when data movement dominates. Sparse FP16 performance reaches 733 TFLOPS on the L40S against 419 TFLOPS on the RTX 5090 so workloads that exploit sparsity obtain higher throughput on the L40S when dense FP16 figures are compared directly with dense FP16 figures.

Choose the L40S when your model needs between 32 GB and 48 GB of VRAM, which is the gap the RTX 5090 cannot cover. It also holds the higher dense FP16 figure at 362.05 TFLOPS against 209.5 TFLOPS, so compute bound work such as fine-tuning at large batch sizes leans its way. As a datacenter card it has ECC memory and is offered in multi-GPU servers, which makes it the safer choice for long training runs you do not want to babysit.

Choose the RTX 5090 when the model fits in 32 GB and speed per GPU matters, since its 1792 GB/s bandwidth is 2.1 times the 864 GB/s of the L40S and directly raises tokens per second during decode. It is also the only one of the two with FP4 tensor support at 1676 TFLOPS dense, which lets FP4 quantized models run natively. Treat it as a consumer card: PCIe 5.0 only, no NVLink and no ECC, so it suits single GPU inference and diffusion work rather than multi-GPU training.