A practical GPU budget for AI startups in 2026

Why GPU budgets fail

GPU spend is usually the largest line in an AI startup's infrastructure bill, and the planning behind it is often a single monthly number nobody can decompose. Two failure patterns account for most overruns. The first is budgeting in dollars instead of work: "we'll spend about this much" gives engineers nothing to reason about. The second is that nobody owns idle time, so a forgotten dev box or a finished training node runs for a week before anyone notices.

This post is a template for planning one quarter. Plan in GPU-hours, which engineers can estimate, then convert to dollars at the end using the current rates on /pricing. Because rates change and differ by SKU, every cost in this post is a formula, not a figure.

Split the budget into three buckets

Different work has different shapes, and each shape suits a different pricing model.

Track each bucket separately all quarter. When spend surprises you, the bucket tells you where to look, and the conversations are different: experimentation overruns are a hygiene problem, training overruns are a planning problem, and inference overruns are usually a sizing or efficiency problem.

On-demand versus reserved

The decision is a single-variable problem. For a GPU you could reserve:

utilization   = GPU-hours actually used / (GPUs x hours in period)
on-demand     = GPU-hours used x on-demand hourly rate
reserved      = months x reserved monthly rate
break-even    = 1 - reserved discount

If a 30% reserved discount is on offer, reserving pays off above 70% utilization. Reserved terms run from one to twelve months, and you can switch a running instance between modes without redeploying. The full reasoning, including the hybrid pattern most teams settle on, is in On-demand vs reserved: choosing a GPU pricing model.

In practice, for a startup:

• Reserve the inference floor, meaning the replicas that run 24/7 regardless of traffic, and any node that genuinely trains every day.
• Stay on-demand for experimentation and for training runs separated by weeks of evaluation. A dev box used five hours a day is at about 21% utilization, far below any break-even.

Per-second billing and idle hygiene

Every Reviosa instance bills by the second, and stopped instances accrue nothing for compute. Volumes and IPs persist, so your work survives a stop, and storage keeps billing at its own monthly rate. Per-second billing only saves money if instances actually stop, which makes idle hygiene the cheapest optimization you have.

Four rules cover most of it:

1. Every instance has an owner. Put the owner's name in the instance name; an unowned instance gets stopped at the weekly sweep.
2. Checkpoint often. If stopping a box loses work, people will not stop boxes. Save checkpoints and environments to a persistent volume.
3. Automate idle detection on dev boxes. A small cron job, shown below, is enough.
4. Sweep weekly. List running instances older than seven days and ask each owner whether it still needs to run.

The idle detector for rule 3:

#!/usr/bin/env bash
# idle-watch.sh: run from cron every 5 minutes on a dev box
THRESHOLD=5     # percent GPU utilization
LIMIT=6         # consecutive idle checks (6 x 5 min = 30 min)
STATE=/var/tmp/idle-count

util=$(nvidia-smi --query-gpu=utilization.gpu --format=csv,noheader,nounits | sort -n | tail -1)
count=$(cat "$STATE" 2>/dev/null || echo 0)
if [ "$util" -lt "$THRESHOLD" ]; then count=$((count + 1)); else count=0; fi
echo "$count" > "$STATE"

if [ "$count" -ge "$LIMIT" ]; then
  echo 0 > "$STATE"
  "$STOP_HOOK"    # your stop action via the API or CLI
fi

Point STOP_HOOK at a stop through the platform API or CLI, not just an OS shutdown, and confirm in the console that the instance shows as stopped.

Budget alerts

Set a monthly budget on the Billing page of the dashboard and turn on spend alerts: you get an email at 75% and at 100% of the budget. Set the budget to your planned monthly spend, not a padded number, so the alerts carry information.

Read the 75% alert against the calendar. If it arrives on day 22 of a 30-day month, you are on plan. If it arrives on day 12, you are running at nearly twice plan, and you have time to find out why before the invoice does it for you. The Usage page itemizes spend per instance, which is usually enough to find the cause in minutes.

A sample quarterly budget

Here is a worked example for a hypothetical team: six ML engineers, one product in production serving an 8B-class model, and two planned fine-tuning projects. Let R_sku be the on-demand hourly rate and M_sku the reserved monthly rate for each SKU from /pricing.

That is about 9,000 GPU-hours for the quarter. A few observations that tend to generalize:

• Inference dominates the hours. Nearly two-thirds of GPU-hours here are serving, which is why serving efficiency matters so much. Quantization and right-sizing, covered in Cutting inference costs 60%, shrink the largest line.
• Training dominates the risk. Fewer hours, but at node rates, and failed runs are common. Budget the rerun buffer explicitly instead of hoping.
• Only one line is an obvious reservation. The inference floor runs at 100% utilization. Everything else is below break-even and belongs on per-second on-demand billing.

To turn the table into a number, fill in a rates map once per quarter:

rates = {"rtx-5090": 0.0, "h100-sxm": 0.0, "h100-sxm-8x": 0.0, "l40s": 0.0}        # on-demand, per hour, from /pricing
reserved_monthly = {"l40s": 0.0}                                                     # reserved, per month, from /pricing

hours = {"rtx-5090": 1890, "h100-sxm": 624 + 156, "l40s": 1300}
node_hours = 36 + 30 + 20

subtotal = sum(hours[k] * rates[k] for k in hours) + node_hours * rates["h100-sxm-8x"]
subtotal += 2 * 3 * reserved_monthly["l40s"]
print(f"Compute subtotal: {subtotal:,.2f}  with 12% contingency: {subtotal * 1.12:,.2f}")

The monthly review ritual

Forty-five minutes, once a month, with whoever owns the budget and one engineer from each bucket:

1. Plan versus actual by bucket, in GPU-hours and in dollars.
2. Utilization of every reserved line. Anything below its break-even is a candidate to release or resize at renewal.
3. Top ten instances by spend. Each owner explains theirs in one sentence.
4. Idle report. Hours that instances ran with near-zero GPU utilization, and which rule failed.
5. Unit economics. Cost per million tokens served, cost per training run, cost per experiment. Trends here matter more than totals.
6. Re-forecast next month and update the budget amount behind the alerts.
7. Decide: reserve, release, resize, or leave alone. Write the decision down.

Larger commitments, such as reserved terms across many GPUs or dedicated HGX pods, are worth a conversation: contact sales for a custom quote, which we turn around within one business day.

FAQ

How much should an AI startup budget for GPUs?

Estimate work first, not dollars. Split the quarter into experimentation, training and inference, estimate GPU-hours for each from team size, planned runs and serving replicas, then multiply by current rates. Add a rerun buffer for training and 10-15% contingency overall.

Should a startup reserve GPUs or use on-demand?

Reserve only what you can keep busy above the break-even utilization, which equals one minus the reserved discount. For most startups that is the 24/7 inference floor. Experimentation and occasional training runs are usually cheaper on per-second on-demand billing.

How do I stop paying for idle GPUs?

Stop instances when they are not working; stopped instances accrue nothing for compute. Give every instance an owner, checkpoint to persistent volumes, automate idle detection on dev boxes, and sweep long-running instances weekly.

How do budget alerts work on Reviosa?

Set a monthly budget on the Billing page of the dashboard and enable spend alerts. You receive an email when spend reaches 75% and 100% of the budget, and the Usage page itemizes spend per instance so you can find the cause.