Which AI model scores highest on NL2Repo-Bench?

DeepSeek-V4.1-Flash by DeepSeek holds the best NL2Repo-Bench result among tracked models, at 65.4% (released Sep 10 2026). Higher scores are better on this benchmark.

What are the top 5 models on NL2Repo-Bench?

1. DeepSeek-V4.1-Flash (DeepSeek) — 65.4%; 2. Qwen3.8-Max-0902 (Qwen) — 64.9%; 3. GLM-5.3-Flash (Z.ai) — 56.3%; 4. Qwen3.8-Max (Qwen) — 55.9%; 5. Qwen3.8-Flash-Next (Qwen) — 48.1%.

How many models have a published NL2Repo-Bench score?

6 tracked models have a published NL2Repo-Bench score. Scores are the figures reported by each lab at that model's release, so this page is a record of results over time rather than a re-run leaderboard.

What is the best open model on NL2Repo-Bench?

GLM-5.3-Flash by Z.ai is the highest-ranked model with downloadable weights on NL2Repo-Bench, scoring 56.3% at rank 3 overall.

Repo-level code generation

NL2Repo-Bench

Tests whether the AI can turn a natural-language requirement into working code across an entire repository, not just produce a single function or patch. Higher is better.

Rankings

Higher is better

NL2Repo-Bench — frequently asked questions

: Tests whether the AI can turn a natural-language requirement into working code across an entire repository, not just produce a single function or patch. Higher is better.
: DeepSeek-V4.1-Flash by DeepSeek holds the best NL2Repo-Bench result among tracked models, at 65.4% (released Sep 10 2026). Higher scores are better on this benchmark.
: 1. DeepSeek-V4.1-Flash (DeepSeek) — 65.4%; 2. Qwen3.8-Max-0902 (Qwen) — 64.9%; 3. GLM-5.3-Flash (Z.ai) — 56.3%; 4. Qwen3.8-Max (Qwen) — 55.9%; 5. Qwen3.8-Flash-Next (Qwen) — 48.1%.
: 6 tracked models have a published NL2Repo-Bench score. Scores are the figures reported by each lab at that model's release, so this page is a record of results over time rather than a re-run leaderboard.
: GLM-5.3-Flash by Z.ai is the highest-ranked model with downloadable weights on NL2Repo-Bench, scoring 56.3% at rank 3 overall.