Robert Zieliński Feeds Hardware Private inference on RTX 5090 - how much saved? — article mark Private inference on RTX 5090 - how much saved? 2026-10-17
Qwen3.8-27B on our traffic: API $1,075.10 vs RTX 5090 at $0.70/h, electricity included, $57.71–$109.07 . Saved: $966–$1,017 (90–95%) .
Rent now — $0.70/h
Our data
Counter Tokens Meaning Σ generated 81.7M output tokens (decode) Σ prompt 1.66B all input tokens sent Σ prefill 225M input actually computed on our GPU; the other 1.435B (86.4%) was reused from the KV cache Rental $0.70/h one RTX 5090, electricity included — the only 5090 cost
Token volumes: prompt 1.66 billion, prefill 225 million, generated 81.7 million
Token volumes (millions; 0.25 px per 1M)
Σ prompt
1,660M
Σ prefill
225M (13.6% of prompt)
Σ generated
81.7M
20.3 prompt tokens per generated token (1.66B ÷ 81.7M); 95.3% of all tokens are input.
Σ counters from our self-hosted Qwen3.8-27B: prompt 1.66B, prefill 225M, generated 81.7M.
Price per 1M tokens
Per 1M tokens Qwen3.8-27B API RTX 5090 @ $0.70/h 5090 throughput Saving Generated (decode) $3.00 $0.68 287.4 tok/s 77% Prompt (prefill) $0.50 $0.011–$0.032 6–18K tok/s 94–98%
5090 $ per 1M = $0.70 ÷ (tok/s × 3,600 ÷ 1M) = 194.44 ÷ tok/s . API: Alibaba Cloud Model Studio , qwen3.8-27b, International (Singapore), page updated 2026-10-15. 5090 throughput: vLLM, Qwen3.8-27B NVFP4, MTP@3, 4 concurrent streams .
Price per million tokens: generated, API 3.00 dollars vs RTX 5090 0.68 dollars; prompt, API 0.50 dollars vs RTX 5090 0.011 to 0.032 dollars
$ per 1M tokens, Qwen3.8-27B (scale $0–3.00)
Generated · API
$3.00
Generated · RTX 5090
$0.68 −77%
Prompt · API
$0.50
Prompt · RTX 5090
$0.011–$0.032 −94–98%
Bar length = $ per 1M tokens (140 px per $1). RTX 5090 fully occupied at $0.70/h.
Our bill: API vs RTX 5090
Tokens Qwen3.8-27B API RTX 5090 hours RTX 5090 @ $0.70/h Generated 81.7M $245.10 79.0 h $55.28 Prompt, all prefilled 1.66B $830.00 25.6–76.9 h $17.93–$53.80 Total, all prompt prefilled $1,075.10 104.6–155.8 h $73.21–$109.07 Prompt, measured prefill only 225M — 3.5–10.4 h $2.43–$7.29 Total, measured prefill only $1,075.10 82.4–89.4 h $57.71–$62.57
Saving vs $1,075.10 $ saved % RTX 5090, all prompt prefilled $966.03–$1,001.89 90–93% RTX 5090, measured prefill only $1,012.53–$1,017.39 94–95%
Bill for our traffic: Qwen3.8-27B API 1,075.10 dollars (830 prompt plus 245.10 generated); RTX 5090 with all prompt prefilled 109.07 dollars at most; RTX 5090 with only the measured 225 million prefilled 62.57 dollars at most
Our bill, USD (0.4 px per $1; RTX 5090 at the slower 6K tok/s prefill)
Qwen3.8-27B API
prompt $830.00
gen $245.10
$1,075.10
5090 · all prefilled
$109.07 −90%
5090 · measured prefill
$62.57 −94%
generated $55.28
prompt $53.80 / $7.29
At the faster 18K tok/s prefill: $73.21 (all prefilled) and $57.71 (measured prefill).
Stacked bars: generated + prompt cost. API at the official qwen3.8-27b price; RTX 5090 = GPU-hours × $0.70.
Per 1M generated, our prompt mix
Our traffic sends 20.3 prompt tokens per generated token (1.66B ÷ 81.7M).
Per 1M generated Qwen3.8-27B API RTX 5090 @ $0.70/h Generated 1M $3.00 $0.68 Prompt 20.3M $10.16 $0.22–$0.66 Total $13.16 $0.90–$1.34 · −90–93%
Limits of this estimate
All-in rate: $0.70/h is the rental price with electricity included; no separate power cost is added.Fully occupied: the 5090 cost assumes every rented hour runs 4 concurrent streams. An idle hour still costs $0.70.Throughput: 287.4 tok/s is the 4-stream decode peak at short context; it falls to 169.2 tok/s at 32K ($1.15 per 1M generated, still −62%).Quantisation: 27B fits a 32 GB 5090 as NVFP4 weights + FP8 KV cache; the API serves the full-precision model.
TL;DR
TL;DR: Qwen3.8-27B per 1M tokens: API $3.00 generated / $0.50 prompt; a fully occupied RTX 5090 at $0.70/h costs $0.68 / $0.011–$0.032. On our traffic (81.7M generated, 1.66B prompt, 225M prefilled) that is $1,075.10 vs $58–$109, a 90–95% saving .