Nemotron Labs Dory 1.0 4BOpen Weight

Nemotron Labs Dory 1.0 4B was released on October 16, 2026 as the first model in NVIDIA's Nemotron Labs Dory line. Where the Nemotron 3 and 3.5 models use a hybrid Mamba-Transformer design, Dory is a 4-billion-parameter looped normalized-GPT: eight-layer input and output blocks around a twenty-layer recurrent block that can be applied again at inference. One checkpoint therefore serves different compute budgets, and NVIDIA recommended two loops as the default. Pretraining ran for 6 trillion tokens, the first 4.5 trillion of them in NVFP4, before the context was extended to 256,000 tokens.

NVIDIA released the weights under OpenMDW 1.1 in BF16, MXFP8, NVFP4 and a Q5_K_M GGUF. English is the supported language. At two loops it reported 72.41% on GPQA Diamond, 59.5% on SWE-Bench Verified, 18.1% on Terminal-Bench 2.1 with the Terminus-2 harness, and 77.94% on IFBench under strict constraint checking. On the equal-weight average of the 12 benchmarks in its technical report, two loops scored 39.67, above the 36.73, 32.59 and 32.21 NVIDIA recorded there for Gemma 4 26B-A4B, Gemma 4 12B and Qwen3.5-9B. One loop scored 35.17 on that average, under the 36.73 recorded for Gemma 4 26B-A4B.