Serve models at the latency your users expect
Run vLLM or SGLang on dedicated GPUs you control, or put a model behind a serverless endpoint that scales with traffic and down to zero when it is quiet.
Recommended start
L40S $1.19/hr
4 regions
across North America, Europe and Asia
0 GPUs
billed while a serverless endpoint is idle
1 second
billing granularity on dedicated GPUs
What makes this workload expensive
The bill is rarely about the GPU price alone. These are the problems we design around.
Spiky traffic
Peak load can be ten times the average. Sizing for the peak means paying for idle GPUs most of the day.
Cost per token
Throughput depends on memory bandwidth, batch size and quantization — the cheapest card per hour is rarely the cheapest per token.
Latency close to users
A global user base needs endpoints near it, not one region on the other side of the world.
Built for the shape of the job
Serverless endpoints
Open-weight models behind autoscaling endpoints. Pay per token, not per idle GPU.
Dedicated inference GPUs
L40S, H100 PCIe and RTX 5090 instances for steady traffic you want to own end to end.
Four regions
US East, US West, EU Central and Asia-Pacific — serve each audience from the closest region.
Usage you can see
Per-second usage and spend stream into the console so cost per request stays visible.
Where to start, where to scale
L40S
Start here48 GB VRAM · 1× NVIDIA L40S
$1.19/hr
H100 PCIe
80 GB VRAM · 1× NVIDIA H100
$3.19/hr
RTX 5090
32 GB VRAM · 1× NVIDIA RTX 5090
$1.29/hr
From first run to production
-
1
Benchmark tokens/second for your model on two or three GPU types.
-
2
Pick serverless for spiky traffic or dedicated GPUs for steady load.
-
3
Deploy in the region closest to your users.
-
4
Watch cost per request in the usage dashboard and right-size.
Guides for this workload
October 13, 2026
Cutting inference cold starts: container images and weights
September 17, 2026
L40S vs H100 vs RTX 5090: inference cost per million tokens
August 7, 2026
Serving MoE models with expert parallelism on 8×H200
Serverless suits spiky or early-stage traffic; dedicated GPUs are usually cheaper for steady, high utilization.
Yes. Dedicated instances run any container or framework you install.
US East (Ashburn), US West (Portland), EU Central (Frankfurt) and Asia-Pacific (Tokyo).
Other workloads
Start training in minutes
Create an account, add a card, and launch your first GPU instance. Per-second billing means you only pay for what you use.
No minimum commitment · Cancel anytime · $10 free credit for new accounts