TL;DR — We trained a DFlash2 block-diffusion speculative-decoding drafter for GLM-5.3-Flash from scratch, against our own W4A16-MTP quant: 350,260 self-generated samples, 35,021 steps, every completion regenerated by the target model itself — no third-party drafter weights or traces anywhere in the training path. On a same-rig A/B (2× DGX Spark, TP=2, K=7, 500-prompt holdout) it beats our previous best dflash2DF2 in every cell (3.5788 acceptance / 70.87 tok/s at c16, +2.2% throughput) and edges the widely-used incoai reference on single-stream acceptance, while trailing its c16 throughput primary by 2.9%. We ship it anyway: it is fully ours, Apache-2.0 — the reference is CC-BY-NC-ND-4.0. It has been our serving drafter-of-record since 15 Sep 2026, with zero stream errors since. Model: canada-quant/GLM-5.3-Flash-DFlash2-E. Serving stack: canada-quant/vllm-glm53-flash-sm121.
Why a drafter, and why our own
Our GLM-5.3-Flash serving stack on 2× DGX Spark runs speculative decoding: a small drafter model proposes a block of candidate tokens, the full target model verifies them in a single forward pass, and accepted tokens are exactly what the target would have produced anyway. It is a lossless speedup — it changes latency, not answers — and on this stack the drafter's acceptance rate is the single biggest interactive-latency lever we have. We use a DFlash2 block-diffusion drafter: rather than drafting one token at a time autoregressively, it denoises a block of 8 candidate positions at once, conditioned on hidden states tapped from the target, giving K=7 speculative tokens per verification pass.
Until this week we served with dflash2DF2, our previous best, and benchmarked against the widely-used incoai reference drafter — which edged our DF2 on throughput. But the reference ships under CC-BY-NC-ND-4.0: non-commercial, no-derivatives. That is not a license a sovereign lab serving regulated industries can build a production path on. The gap was small enough to chase — so we trained our own, end to end, on data we control.
What we did
The artifact is dflash2E — released as canada-quant/GLM-5.3-Flash-DFlash2-E (Apache-2.0), a ≈1.84B-parameter drafter in a 6.2 GB bf16 checkpoint, trained against our own quant canada-quant/GLM-5.3-Flash-W4A16-MTP as the target. The training run: 350,260 self-generated samples, 35,021 steps, lr 1e-4, warm-started from our own previous best drafter dflash2DF2.
The part that matters to us is provenance. Every training completion was regenerated by the target model itself — no third-party drafter weights, no third-party traces, nothing in the training path we cannot audit and re-run. The prompt sources are all openly licensed: ultrachat_200k (MIT), OpenR1-Math-220k (Apache-2.0), OpenCodeReasoning/OpenCodeInstruct (CC-BY-4.0), and evol-codealpaca-v1 (Apache-2.0). The result is a drafter that is reproducible end-to-end and unencumbered: Apache-2.0, no NC/ND terms.
Architecture
Eight decoder layers with full attention, hidden size 4096. The drafter conditions on 9 hidden-state taps on the target at layers [5, 9, 14, 19, 24, 28, 33, 38, 42], denoises a block of 8 positions per pass — K=7 speculative tokens — and supports max positions of 1,048,576, matching the target's 1M-token context.
The A/B, honestly
Same-rig A/B on our production configuration: 2× DGX Spark (SM121), TP=2, K=7, a 500-prompt holdout, concurrency 16 as the primary cell. Numbers are acceptance length and output tok/s.
| Drafter | Acceptance @ c16 | tok/s @ c16 | License |
|---|---|---|---|
| dflash2E (this) | 3.5788 | 70.87 | Apache-2.0 |
| dflash2DF2 (our previous best) | 3.5020 | 69.33 | Apache-2.0 |
| incoai reference | 3.6274 | 72.98 | CC-BY-NC-ND-4.0 |
Against dflash2DF2, dflash2E wins every cell — +2.2% throughput on the c16 primary (70.87 vs 69.33 tok/s) with acceptance up 3.5020 → 3.5788. Against the incoai reference, it loses the c16 throughput primary by 2.9% (70.87 vs 72.98) — we say that plainly — while edging it on single-stream acceptance: 3.5970 vs 3.5760. Relative to the gap DF2 had left open to the reference, dflash2E closed 61% of it.
We ship it anyway, and the reasons are the point of this lab: it is fully ours (Apache-2.0 — the reference's NC/ND terms exclude it from commercial serving paths like ours), it is trained on data we control, and it is reproducible end-to-end. A 2.9% throughput deficit we can attack with another training round; a license we cannot renegotiate is a wall.
Cross-hardware fidelity
Acceptance behaviour is a property of the drafter/target pair, not of the rig — and it should reproduce off the training hardware. On an H200 holdout it holds: 3.568 acceptance / 1,277.5 tok/s at c16 — the acceptance profile reproduces across SM121 and H200 within 0.3%.
Adoption: drafter-of-record
We adopted dflash2E on 15 Sep 2026 as the lab's serving drafter-of-record on the 2× DGX Spark (SM121) stack, superseding dflash2DF2. The stack has served continuously since, with zero stream errors. The checkpoint was downloaded 775 times in its first three days on Hugging Face.
The serving stack is open too
A drafter is only useful if you can serve it, so we open-sourced the whole thing: canada-quant/vllm-glm53-flash-sm121 (created 17 Sep 2026, Apache-2.0) — one-command serving of GLM-5.3-Flash W4A16 + a DFlash2 drafter on 2× NVIDIA DGX Spark (GB10, SM121a), tensor-parallel over RoCE. A prebuilt aarch64 image (ghcr.io/canada-quant/vllm-glm53-flash-sm121:v2-w4a16-dflash2e) carries the stack; the drafter is bind-mounted, so drafters swap without a rebuild.
The production config: TP=2 + expert-parallel, 262K context (1M possible), fp8_e4m3 KV cache, K=7, CUDA graphs FULL_AND_PIECEWISE, and a 366,749-token KV pool. Two serving-critical patches are baked into the image, sha256-gated at build time.
Limitations (honestly)
- Still behind on the primary: the incoai reference keeps the c16 throughput primary by 2.9% (72.98 vs 70.87 tok/s). Its CC-BY-NC-ND-4.0 license keeps it out of our serving path regardless — but the gap is real, and closing it is on us, not on their license.
- One target: dflash2E is trained against our W4A16-MTP quant of GLM-5.3-Flash. Acceptance against other targets (other quants, the BF16 base) is not characterized here.
- Early adoption: 775 downloads in the first three days is a start, not validation. Independent reproductions are what will prove the card out.
What's next
- Close the 2.9%: a second training round aimed at the c16 throughput primary, same self-generated-data recipe, is the direct follow-up.
- 1M-context serving: the production config runs 262K context today; the stack supports 1M, and dflash2E's 1,048,576 max positions are sized for it.