Blackwell B200 vs Hopper: what changes when you upgrade

The short version

The jump from Hopper to Blackwell is not the H100-to-H200 story again. H200 was the same GH100 compute die with a bigger, faster memory system bolted on. B200 is a new architecture: two reticle-sized dies joined by a 10 TB/s chip-to-chip link and presented to software as one GPU, fifth-generation tensor cores with new low-precision formats, NVLink 5, and a power envelope that is meaningfully higher. Some workloads get close to the headline gains. Many get a solid but smaller improvement. A few are better left on Hopper.

This post walks through what changes, dimension by dimension, using NVIDIA's public spec-sheet figures. Where we give speedup ranges, they are approximate expectations for planning, not our lab measurements. Your own benchmark is the only number that matters.

Spec sheet, side by side

All compute figures below are dense (no structured sparsity), per GPU, rounded from NVIDIA's published SXM specifications. Board power is the maximum configurable TDP; system integrators may run lower.

Three ratios are worth memorizing: about 2.3× the dense BF16 and FP8 math of an H100, about 1.7× the memory bandwidth of an H200, and 2× the NVLink bandwidth. Everything below follows from which of those three your workload is actually limited by.

Memory: capacity and bandwidth

At 180 GB, a single B200 holds what used to require sharding. A 70B model in FP8 is roughly 70 GB of weights, which leaves over 100 GB for KV cache on one GPU. On an H100 the same model in FP8 barely fits with almost no room for context; in BF16 (about 140 GB) it does not fit at all.

For decode-heavy inference, bandwidth is the number that matters. Generating one token requires streaming every active weight through the tensor cores once, so tokens per second per sequence at small batch sizes scales close to linearly with memory bandwidth. Going from 4.8 TB/s to roughly 8 TB/s is the single most predictable gain in the upgrade: expect decode at low batch to land somewhere around 1.4-1.7× an H200 before any precision change, depending on how much time is spent outside the weight-streaming kernels. If you have read our earlier piece on why VRAM, not FLOPS, limits your batch size, the logic is the same: capacity sets your maximum batch, bandwidth sets your per-token floor.

For training, extra capacity usually buys you less recomputation rather than raw speed. Fewer activation-checkpointed layers, larger micro-batches, or one less level of model parallelism. Those are real gains, but they are workload-specific, so measure them.

Precision: FP8 goes mainstream, FP4 arrives

Hopper introduced FP8 tensor cores with per-tensor scaling, managed in software by Transformer Engine. Blackwell keeps FP8 and adds two things:

• Hardware block scaling (MX formats). MXFP8 stores a shared 8-bit power-of-two scale for every 32 elements, applied by the tensor core itself. Finer-grained scaling means fewer overflow and underflow problems than per-tensor FP8, which in practice makes FP8 training less fiddly. We cover the recipes in our FP8 training guide.
• FP4 (NVFP4 and MXFP4). Roughly double the FP8 throughput on paper, aimed primarily at inference. Weight-and-activation FP4 needs calibration and careful evaluation; weight-only FP4 is the easier first step.

The practical implication: the biggest B200 speedups in public benchmarks come from combining the new hardware with a lower precision than you ran on Hopper. If you move a BF16 Hopper workload to B200 and keep it in BF16, you are leaving the most compelling part of the architecture unused. If you move an FP8 Hopper workload to FP4 on B200, you are changing two variables at once and must re-validate quality.

NVLink 5 and multi-GPU topology

NVLink 5 doubles per-GPU bandwidth to 1.8 TB/s. Within an 8-GPU HGX B200 baseboard, that makes tensor parallelism across all eight GPUs cheaper, and it speeds up the all-to-all traffic that mixture-of-experts models generate during expert-parallel inference (see our guide to serving MoE models with expert parallelism).

What NVLink 5 does not change is the cost of going between nodes. Scale-out still runs over InfiniBand or Ethernet, and the same NCCL hygiene applies; our InfiniBand tuning guide carries over. In fact, a faster intra-node fabric makes the inter-node hop relatively more expensive, so jobs that were borderline communication-bound on Hopper can become more so on Blackwell unless you overlap communication with compute aggressively.

On Reviosa, the B200 SXM is available as a single-GPU instance in the GPU cloud fleet. If you need multiple B200s in one NVLink domain for a training run, talk to us about a reserved HGX pod through sales.

Power and density

A B200 can draw up to roughly 1,000 W, against 700 W for H100 and H200. Per unit of work it is considerably more efficient, but per GPU-hour it consumes more, which is one reason B200 instances are priced above Hopper everywhere in the market. When you compare, compare cost per unit of output (tokens per dollar, steps per dollar), not cost per GPU-hour. Current rates for every SKU are on our pricing page.

Software stack readiness

Blackwell datacenter GPUs are compute capability 10.0 (sm_100). That has concrete consequences:

• CUDA 12.8 or newer is required to compile for sm_100. Our default images ship PyTorch 2.7 on CUDA 12.8, which is the first PyTorch release line with official Blackwell wheels.
• Custom kernels need rebuilding. Any extension compiled only for sm80 or sm90 will either fail to load or fall back to PTX JIT, which is slow at startup and rarely optimal.
• Hopper-specific kernels do not transfer. FlashAttention-3 was written around Hopper's warpgroup MMA instructions. On Blackwell, use attention backends that ship sm_100 kernels, such as cuDNN through PyTorch SDPA, FlashInfer, TensorRT-LLM, or newer FlashAttention releases that explicitly list Blackwell support.
• Consumer Blackwell is different. The RTX 5090 is sm120. A wheel built for sm100 does not automatically cover it, and vice versa.

A quick readiness check before you migrate a training or serving image:

python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.get_device_capability())"
# Expect CUDA 12.8+ and (10, 0) on B200
python -c "import torch; print(torch.cuda.get_arch_list())"
# sm_100 should appear in the list

Then run your actual model for a few hundred steps or a few thousand requests and compare loss curves or output quality against Hopper before switching production traffic.

Where the gains show up

As a planning heuristic, not a promise:

When not to upgrade

• Your GPU is mostly idle. If profiling shows the GPU waiting on data loading, tokenization, Python overhead, or network I/O, a faster GPU makes the waiting more expensive. Fix utilization first.
• The model fits comfortably and latency is fine. A 7B or 13B model serving moderate traffic within its latency SLO on an H100 or L40S gains little from B200 headroom.
• Your stack is pinned. If you depend on a framework version, custom CUDA extension, or vendor library without sm_100 support, budget engineering time for the port before budgeting GPU hours.
• You need many GPUs across nodes today. If the job is dominated by inter-node communication, the faster chip shifts the bottleneck rather than removing it.
• The math does not close. If your measured speedup is smaller than the price ratio, Hopper is the cheaper way to get the work done. That calculation changes per workload; do it per workload.

How we would approach a migration

Start with inference, where bandwidth gains are predictable and rollback is trivial. Benchmark your production prompt mix on one B200 against your current H100 or H200 deployment at matched precision, then try one precision step down. For training, port the image, confirm sm_100 kernels are in use with a profiler trace, and run a short loss-curve comparison before committing a long run. Because billing is per second, a thorough comparison costs a few GPU-hours, which is cheap insurance against a multi-week commitment to the wrong hardware.

FAQ

Is B200 always faster than H200?

For compute-bound and bandwidth-bound GPU work, yes, by a wide margin on paper. For workloads limited by CPU, data loading, or small kernels, the difference can be negligible, so profile before you assume.

Do I need to change my code to run on B200?

Usually not for PyTorch models that use standard ops, as long as you run CUDA 12.8+ and a Blackwell-aware PyTorch build. Custom CUDA extensions and Hopper-only kernels such as FlashAttention-3 need rebuilding or replacement.

Should I use FP4 on B200?

FP4 is mostly an inference format today. Weight-only FP4 is a reasonable first experiment; full FP4 for weights and activations requires calibration and careful evaluation against your quality metrics.

How do I compare B200 and Hopper costs fairly?

Measure throughput on your workload, then divide by the hourly rate on our pricing page to get cost per token or per training step. Comparing per GPU-hour alone hides the performance difference.