← Inference Inference · Qwen 4 architecture

Qwen3.8 Flash Next,
the Qwen 4 architecture, today.

Qwen’s official preview of the architecture that will underpin Qwen 4 — Gated DeltaNet plus Qwen Sparse Attention, 512 experts. The FP8 weights fit two DGX Sparks wired into one 256 GB pool, served with vLLM behind an OpenAI-compatible /v1 endpoint.

Which machine fits

The model from the published weights
Developer
Alibaba Qwen
Released
August 24, 2026
Parameters
179,999,981,459 (180.0B) total · 6B active per token
Licence
Qwen Community 1.0
Context
262,144 tokens native · up to 1,000,000 with YaRN
Architecture
48 layers · 12 × (3 Gated DeltaNet + 1 Qwen Sparse Attention)
Attention
QSA · 24 query / 2 KV heads · head dim 256 · block indexer
Experts
512 experts · 10 routed + 1 shared per token
Draft
1 MTP layer · speculative decoding built in
Input
text, image, video
Weights on Hugging Face →
Which machine fits tokens of KV cache left
Weights Size DGX Spark · 128 GBH200 · 141 GB2× DGX Spark · 256 GB
BF16 360.00 GB does not fitdoes not fitdoes not fit
FP8 185.52 GB does not fitdoes not fit1,638,4346 × full-context sequences

Memory vLLM takes (90% of the machine) minus the weights, divided by 27,392 B of BF16 KV cache per token (K + V of 12 QSA layers and 1 MTP layer × 2 KV heads × 256 × 2 B, plus the indexer’s 128-dim keys at 1 per 4 tokens). Each sequence also holds 115,458,048 B of fixed Gated DeltaNet state. One full 262,144-token conversation is one sequence. No official NVFP4 build exists yet.

How it is served one tenant per pool
Endpoint
OpenAI-compatible /v1 — chat, completions, streaming.
Engine
vLLM, official FP8 weights, tensor-parallel over both Sparks.
Machine
Two DGX Sparks wired together as one 256 GB pool.
Tenancy
One customer per pool — nobody else’s requests run on it.
Location
Poland, EU — on machines we own and run.
Billing
Per hour, USD + VAT. Rate set once the service opens.

Run the Qwen 4 architecture on your own pool.

Write what load you expect and how much context you need. Reply within one business day.

hello@robertz.co

Not open yet

We are checking the demand.

We are collecting interest to see whether there is demand to open this service. Let us know you want it, so we open sooner :)

Let us know