If your model fits on one GPU, ignore all of this. NVLink, SXM and HGX describe how GPUs talk to each other, and a single-GPU job never uses that path. Rent the cheapest variant with enough memory and move on.
The interconnect starts to matter when one model is split across several GPUs. It matters most for tensor parallelism across four or eight GPUs and for multi-node training. This page gives the numbers, explains the names on rental listings, and ends with a table that maps your workload to the link you need.
The bandwidth ladder
NVIDIA quotes interconnect bandwidth as a bidirectional total: both directions added together, across all links on the GPU. The bandwidth in one direction is half the headline. That is why some server makers print a smaller number for the same part. Lenovo's B200 product guide (updated 13 January 2026) lists NVLink at 900 GB/s, which is the per-direction half of NVIDIA's 1,800 GB/s.
NVIDIA's PCIe figures follow the same convention, so the rows below compare like with like. All figures were read from the vendor pages on 21 September 2026. The per-direction column is our arithmetic (half the total).
| Link | Vendor figure, both directions | One direction | Where you meet it |
|---|---|---|---|
| PCIe Gen4 x16 | 64 GB/s | about 32 GB/s | L40S, A100 PCIe |
| PCIe Gen5 x16 | 128 GB/s | about 64 GB/s | H100, H200 host link |
| NVLink 2 | 300 GB/s | 150 GB/s | V100 SXM2 |
| NVLink 3 | 600 GB/s | 300 GB/s | A100 SXM4, A100 PCIe bridged pair |
| NVLink bridge, H100 NVL pair | 600 GB/s | 300 GB/s | Two H100 NVL cards |
| NVLink 4 | 900 GB/s | 450 GB/s | H100 and H200 SXM5, H200 NVL bridge |
| NVLink 5 | 1,800 GB/s | 900 GB/s | B200 on an HGX board, GB200 NVL72 |
Two ratios are worth keeping in your head. NVLink 4 carries about 7.0 times the bandwidth of PCIe Gen5 x16 (900 against 128 GB/s). NVLink 5 carries about 14.1 times. NVIDIA's H100 NVL product brief (March 2024) makes a similar claim for the bridged pair: 600 GB/s is "10x the bandwidth of PCIe Gen4".
We have left NVLink 6 out of the table on purpose. On 21 September 2026 NVIDIA's NVLink page gave 3 TB/s per Vera Rubin GPU and its HGX page gave 3.6 TB/s. Both are marked preliminary, and we do not know which will ship.
A per-GPU figure also hides topology. On an HGX board an NVSwitch gives every GPU the full rate to any other GPU. Without a switch, NVIDIA's developer blog (12 August 2024) notes that each of eight GPUs must split the same 900 GB/s into seven point-to-point connections of 128 GB/s each. In NVIDIA's own Llama 3.1 70B test in that post, the switched system matched the point-to-point one at batch size 1 and delivered 1.5 times the throughput at batch size 32. That is vendor data. The switch earns its keep under concurrent load, not for one user at a time.
The site's spec table shows which links each GPU family supports.
| Spec | A100 | H100 | H200 | B200 | L40S | RTX 4090 |
|---|---|---|---|---|---|---|
| VRAM | 40 to 80 GB | 80 to 94 GB | 141 GB | 180 to 192 GB | 48 GB | 24 GB |
| Memory bandwidth | 2,039 GB/s | 3,350 GB/s | 4,800 GB/s | 8,000 GB/s | 864 GB/s | 1,008 GB/s |
| Interconnect | NVLink, PCIe 4.0, InfiniBand | NVLink, PCIe 5.0, InfiniBand | NVLink, PCIe 5.0, InfiniBand | NVLink, PCIe 6.0, InfiniBand | PCIe 4.0 | PCIe 4.0 |
The form factors behind the listing names
SXM
SXM is NVIDIA's proprietary socketed module format. The GPU is a module that sits on a baseboard, not a card in a PCIe slot. Wikipedia's SXM article (read 21 September 2026) describes it as "a high bandwidth socket solution for connecting Nvidia Compute Accelerators to a system". NVIDIA does not publish an expansion of the acronym, so we do not give one.
The generation number tells you the GPU. Per the same article: SXM2 and SXM3 are the V100, SXM4 is the A100, SXM5 is the H100 and H200, and SXM6 is the B100 and B200. Dell's PowerEdge XE9780 listing (read 21 September 2026) also labels the B300 as SXM6. NVIDIA's own pages use "A100-SXM4", "H100-SXM5" and "H200-SXM5", but for Blackwell they say only "Blackwell SXM". The "SXM6" label comes from Dell, Lenovo, HPE and Wikipedia.
The socket matters for two reasons. It feeds more power: the H100 SXM is rated at up to 700 W. And it is the only way to get full NVLink to every other GPU in the box. NVIDIA's A100 datasheet shows the gap within one chip. The A100 SXM runs at 400 W with 2,039 GB/s of memory bandwidth and full NVLink through the HGX board. The A100 PCIe card has a lower power limit, slightly lower memory bandwidth, and NVLink only through a bridge to one other card. So the SXM version has a small edge even on one GPU, but not one that justifies a large premium on its own.
HGX, DGX and MGX
HGX is the GPU baseboard NVIDIA sells to server makers. NVIDIA describes it as a single baseboard with eight SXM modules, and Wikipedia notes that most integrators use prebuilt HGX boards in four or eight socket configurations.
DGX is NVIDIA's own complete server built around that board. The DGX B200, for example, pairs eight Blackwell GPUs and two NVSwitch chips with two Intel Xeon CPUs in one NVIDIA-branded chassis.
MGX is a reference design, not a product you rent. NVIDIA calls it "an open modular reference architecture" for partners, and it is where PCIe parts such as the H200 NVL ship, in systems with up to eight GPUs.
For a renter, HGX against DGX rarely matters. Both put eight SXM GPUs on the same NVLink fabric. What differs between hosts is everything around the board: CPUs, storage and the network between nodes.
H100 NVL and other bridged PCIe cards
The H100 NVL is a PCIe card, not an SXM module. Per NVIDIA's product brief (March 2024) it is a dual-slot PCIe Gen5 card with 94 GB of memory, against 80 GB on the standard H100. It uses three NVLink bridges to connect to "a single adjacent NVIDIA H100 NVL card" at 600 GB/s bidirectional. All three bridges are required, and NVIDIA recommends that the pair sits under the same CPU or PCIe switch.
Exactly one other card is the key point. NVIDIA markets the bridged pair as 188 GB for models up to 70B parameters. A four-card H100 NVL server is therefore two NVLink pairs, and traffic between the pairs crosses PCIe. Databricks made the same warning in its inference guide (12 October 2023): some servers "pair GPUs and have lower bandwidth connections between pairs".
Other PCIe cards bridge too. The A100 PCIe takes an NVLink bridge for two GPUs at 600 GB/s, and the product brief says the plain H100 PCIe card uses the same bridge part. The H200 NVL goes further with a 2-way or 4-way bridge at 900 GB/s per GPU, so up to four cards share NVLink.
A bridge is an optional part. A listing for "2x H100 PCIe" does not tell you whether the host fitted one. Ask, or run nvidia-smi topo -m on the instance and check whether the GPU pairs show an NVLink connection.
Cards with no NVLink at all
NVIDIA's L40S specification has the row "NVIDIA NVLink Support: No", and the card connects over PCIe Gen4 x16 only. The RTX 5090 spec page lists no NVLink, and its bus is PCIe 5.0 x16. Our spec table lists the RTX 4090 as PCIe 4.0 only. The RTX PRO 6000 Blackwell product page does not mention NVLink either.
These cards still work in multi-GPU servers. They just do it over PCIe, which changes how you should split the model.
What the form factor costs right now
The same chip is listed at different prices depending on its form factor. The live tables below show the cheapest current listing for each H100 and A100 variant. The gap between the PCIe and SXM rows is the SXM premium today.
| Variant | VRAM | Cheapest $/GPU-hr | Provider | Providers in stock |
|---|---|---|---|---|
| H100 PCIe | 80 GB | $2.50 | Hyperstack | 7 |
| H100 SXM5 | 80 GB | $2.79 | Lyceum | 7 |
| H100 NVL | 94 GB | $3.11 | Massed Compute | 1 |
| Variant | VRAM | Cheapest $/GPU-hr | Provider | Providers in stock |
|---|---|---|---|---|
| A100 PCIe 80GB | 80 GB | $0.68 | LeaderGPU | 7 |
| A100 SXM4 80GB | 80 GB | $0.80 | Vast.ai | 4 |
| A100 PCIe 40GB | 40 GB | $1.59 | RunPod | 2 |
| A100 SXM4 40GB | 40 GB | $1.99 | Lambda Labs | 1 |
Read the gap per GPU, then multiply by the number of GPUs you need. If the NVL row is close to the PCIe row, the NVL is the better single-GPU buy because of its extra 14 GB of memory. You can see every listing on the H100 rent page and the A100 rent page, and the generational comparison at A100 vs H100.
How the split decides the link
Tensor parallelism cuts every layer across the GPUs, so they exchange data inside every layer of every token. NVIDIA's developer blog (12 August 2024) puts a number on it: a single Llama 3.1 70B query with 8K input tokens and 256 output tokens needs up to 20 GB of synchronisation data transferred from each GPU. While a GPU waits for that data its cores sit idle. This is the workload NVLink was built for.
Pipeline parallelism places whole groups of layers on each GPU and passes activations along only at the boundaries. It moves far less data. The vLLM documentation (read 21 September 2026) is direct about the consequence: "if the GPUs on the node do not have NVLINK interconnect (e.g. L40S), leverage pipeline parallelism instead of tensor parallelism for higher throughput and lower communication overhead".
vLLM's wider rule is a good default. If the model fits on a single GPU, distributed inference is probably unnecessary. If it fits in one node, use tensor parallelism. If it is larger than a node, combine tensor parallelism inside the node with pipeline parallelism between nodes.
Data-parallel training sits in between. Each GPU holds a full copy of the model and the GPUs exchange gradients once per step, not inside every layer. That tolerates PCIe far better than tensor parallelism does. We have no measured figure for this, so time a few steps before you commit to a long run. Once the model or optimizer state has to be sharded across GPUs because it no longer fits on one, communication rises and the job behaves more like the tensor-parallel case.
Before paying for more GPUs, check whether a lower precision removes the split. A 4-bit or 8-bit copy of the model may fit on one card. The LLM VRAM calculator does that arithmetic, and the best GPU for LLM guide lists the usual pairings.
Between nodes: InfiniBand and RoCE
NVLink stops at the edge of the server, or the rack in an NVL72 system. Between nodes, traffic runs over InfiniBand or RDMA over Converged Ethernet (RoCE). NVIDIA's Quantum-2 InfiniBand runs at up to 400 Gb/s per port and Quantum-X800 at 800 Gb/s. Spectrum-X is NVIDIA's Ethernet alternative.
Both fabrics work at scale. Meta wrote on 12 March 2024 that it built two clusters of 24,576 GPUs, one on RoCE and one on Quantum-2 InfiniBand, both with 400 Gbps endpoints, and that it trained Llama 3 on the RoCE cluster.
The step down from NVLink is large. NVIDIA's HGX page lists an HGX B200 at 1.8 TB/s of NVLink per GPU and 0.8 TB/s of networking for the whole eight-GPU node. That is 0.1 TB/s per GPU, an 18 times gap by our arithmetic. Keep tensor parallelism inside the node and send only pipeline or data-parallel traffic across the network. For multi-node quotes, ask each provider which fabric it uses and at what speed per GPU.
Decision table
| Workload | Interconnect you need | What to rent |
|---|---|---|
| Single-GPU inference or fine-tuning | None. The GPU-to-GPU link is never used. | The cheapest variant with enough memory. PCIe is fine. |
| Tensor-parallel inference of a model that does not fit one GPU | NVLink between every GPU in the group. Two GPUs: a bridged NVL pair is enough. Four or eight GPUs: SXM on an HGX board with NVSwitch. | SXM for four or more GPUs. H100 NVL or H200 NVL for two. On L40S or RTX cards, use pipeline parallelism instead. |
| Data-parallel fine-tuning, model fits on each GPU | PCIe is workable. NVLink helps as the model grows and once you shard optimizer state. | PCIe or NVL variants if they are clearly cheaper. SXM if you shard a large model. |
| Large multi-node training | NVLink with NVSwitch inside each node, plus InfiniBand or RoCE between nodes at 400 Gb/s class or better. | Eight-GPU SXM nodes on a reserved cluster. Start at the GPU cluster page. |
The rule in one line: pay the SXM premium only when GPUs must talk to each other inside every layer, which means tensor parallelism across more than two GPUs or sharded training of a large model. For everything else, take the cheaper form factor and spend the difference on memory or hours. Full specifications for every family are in the GPU spec reference, and the H100 vs H200 comparison covers the memory side of the same decision.
Sources
Source pages were retrieved with automated research tools on the dates shown and each figure was traced back to its source before publishing. Rental prices on this page are not typed: they are read live from GPUperhour's own data.
All accessed 21 September 2026 unless a publication date is shown.
- NVIDIA NVLink and NVLink Switch
- NVIDIA HGX platform
- NVIDIA DGX B200
- NVIDIA MGX
- NVIDIA A100 datasheet (PDF)
- NVIDIA V100 datasheet (PDF)
- NVIDIA H100 product page
- NVIDIA H100 NVL product brief PB-11773-001 (PDF), March 2024
- NVIDIA H200 product page
- NVIDIA L40S product page
- NVIDIA GeForce RTX 5090 product page
- NVIDIA RTX PRO 6000 Blackwell product page
- NVIDIA MIG user guide, supported GPUs, updated 11 September 2026
- NVIDIA GB200 NVL72
- NVIDIA developer blog: NVLink and NVSwitch supercharge LLM inference, 12 August 2024
- NVIDIA Quantum-2 InfiniBand
- NVIDIA Quantum-X800 InfiniBand
- NVIDIA Spectrum-X
- Wikipedia: SXM (socket)
- Lenovo Press: ThinkSystem NVIDIA B200 180GB 1000W GPU, updated 13 January 2026
- Dell PowerEdge XE9780
- vLLM documentation: parallelism and scaling
- Databricks: LLM inference performance engineering best practices, 12 October 2023
- Meta Engineering: Building Meta's GenAI infrastructure, 12 March 2024