Specifications Compared
| Spec | Gaudi 2 | L40 |
|---|---|---|
| TDP | 600W | 300W |
| VRAM | 96 GB | 48 GB |
| CUDA Cores | Not published | 18,176 |
| FP8 (dense) | Not published | 362 TFLOPS |
| Memory Type | HBM2e | GDDR6 |
| Architecture | Gaudi | Ada Lovelace |
| FP16 (dense) | Not published | 181.05 TFLOPS |
| Form Factors | OAM | PCIe |
| INT8 (dense) | Not published | 362 TOPS |
| Interconnect | RoCE Ethernet, PCIe 4.0 | PCIe 4.0 |
| Tensor Cores | Not published | 568 |
| FP32 Performance | Not published | 90.5 TFLOPS |
| Memory Bandwidth | 2,450 GB/s | 864 GB/s |
| FP8 (with sparsity) | Not published | 724 TFLOPS |
| FP16 (with sparsity) | Not published | 362.1 TFLOPS |
| INT8 (with sparsity) | Not published | 724 TOPS |
Performance Analysis
The L40 states FP16 dense performance at 181.05 TFLOPS and FP32 performance at 90.5 TFLOPS. The ratio between these dense figures shows how reduced precision increases throughput for training and inference steps. The Gaudi 2 provides no published dense FP16 value for similar comparison.
Current On-Demand Offers
Cheapest secure on-demand offer per provider that is reported in stock right now, per GPU per hour; multi-GPU instances show the whole-instance price alongside. Deploy opens the DeployGPU console for providers on the platform, otherwise the provider's own site. Stock is rechecked every minute.
Gaudi 2
Gaudi 2 is not offered on-demand by any provider we track right now. See the Gaudi 2 rental page for last-seen listed prices and a price alert.
L40
| Provider | Region | GPUs | Per GPU / hr | Instance / hr | Deploy |
|---|---|---|---|---|---|
| ThunderCompute | USA | 1 | $0.79 | — | Deploy |
| RunPod | global | 1 | $0.82 | — | Deploy |
| Massed Compute | us-central-2 | 1 | $0.86 | — | Deploy |
| QuantaCloud | us-midwest-2 | 4 | $0.94 | $3.75 | Deploy |
| Hyperstack | CANADA-1 | 4 | $1.00 | $4.00 | Deploy |
5 providers in stock, 16 offers (cheapest per provider shown). All L40 offers, price history and alerts
Notify me when Gaudi 2 drops below a price
One email when the cheapest in-stock on-demand price per GPU-hour falls below your threshold.
QuantaCloud
Comparing providers? We broker across all of them.
Stop tab-switching between pricing pages. Tell us what you need, 16+ GPUs reserved or cluster capacity, and we return one quote at partner rates within 24 hours.
When to Choose the Gaudi 2
The Gaudi 2 suits workloads that require 96 GB memory capacity and 2450 GB/s bandwidth to accommodate large models or extended batch sizes. Its RoCE Ethernet interconnect further supports distributed setups that benefit from the higher memory and transfer specifications.
When to Choose the L40
The L40 suits workloads that require published dense FP16 performance at 181.05 TFLOPS or dense FP8 performance at 362 TFLOPS within a 300W TDP envelope. Its PCIe form factor also fits standard server configurations that prioritize measured tensor throughput values.
Use Cases
The Gaudi 2 supplies 96 GB memory and 2450 GB/s bandwidth to handle larger models and batches.
The L40 supplies published dense FP16 performance at 181.05 TFLOPS for quantified inference throughput.
The Gaudi 2 supplies 96 GB memory and 2450 GB/s bandwidth to support extended fine tuning sessions.
The L40 supplies published dense FP16 performance at 181.05 TFLOPS within a 300W TDP limit.
The L40 supplies published FP32 performance at 90.5 TFLOPS for precision sensitive calculations.
Frequently Asked Questions
What is the TDP of the L40?▾
The L40 operates at a 300W TDP. The Gaudi 2 operates at a 600W TDP.
What dense FP16 performance does the L40 deliver?▾
The L40 delivers 181.05 TFLOPS in dense FP16 configuration. It also delivers 90.5 TFLOPS in FP32 configuration.
What memory bandwidth does the Gaudi 2 provide?▾
The Gaudi 2 provides 2450 GB/s bandwidth. The L40 provides 864 GB/s bandwidth.
Which interconnect options exist on the Gaudi 2?▾
The Gaudi 2 supports RoCE Ethernet and PCIe 4.0 interconnect. The L40 supports only PCIe 4.0 interconnect.
What dense FP8 performance does the L40 deliver?▾
The L40 delivers 362 TFLOPS in dense FP8 configuration. It also delivers 724 TFLOPS in FP8 with sparsity configuration.
Which is cheaper to rent, the Gaudi 2 or the L40?▾
Cloud rental prices for both the Gaudi 2 and L40 vary by provider, configuration, and availability. This page shows live pricing from 25+ providers updated every 60 seconds. Scroll to the Live Cloud Pricing section to compare current rates.
How much VRAM does the Gaudi 2 have compared to the L40?▾
The Gaudi 2 has 96 GB of HBM2e memory. The L40 has 48 GB of GDDR6 memory.
Can I find Gaudi 2 and L40 GPUs available to rent right now?▾
Yes. This page shows real-time availability across 25+ cloud GPU providers. The Live Cloud Pricing section displays only in-stock offers with current pricing.
What is the main difference between the Gaudi 2 and the L40?▾
The Gaudi 2 uses the Gaudi architecture (2022) while the L40 uses Ada Lovelace (2023). The Gaudi 2 has 96 GB of HBM2e at 2,450 GB/s; the L40 has 48 GB of GDDR6 at 864 GB/s, so the Gaudi 2 has 2.8x the memory bandwidth of the L40.
Rent these GPUs
Each GPU page lists every current on-demand offer by provider, per GPU-hour, updated every minute.
Related comparisons
How this page is made
- Specifications come from the NVIDIA, AMD and Intel datasheets for the Gaudi 2 and the L40. Dense and sparse throughput are listed separately.
- Prices and availability are live: every offer shown is an in-stock, on-demand listing from a provider we track, rechecked every minute.
- The written comparison (overview, performance notes, when to choose each card, verdict, use cases and FAQ) was drafted with an AI model from the specification table above and passed an automated check that rejects any figure not in that table. It was last generated on . It contains no prices; those are always read live.
- Read how we collect the data, or report an error on this page.
Next steps
- Rent GAUDI2Every current offer by provider, daily price history and a price alert.
- Rent L40Every current offer by provider, daily price history and a price alert.
- GPU price indexHow on-demand prices for the major GPUs have moved, updated daily.
- All GPUsEvery GPU we track, with current lows and provider counts.