# Qwen1.5-MoE

Qwen1.5-MoE is an AI model released by Qwen on Mar 28 2024. It has 14.3B parameters, a 32k context window and open weight.

## Facts

| Field | Value |
| --- | --- |
| Model | Qwen1.5-MoE |
| Developer | Qwen |
| Release date | Thursday, Mar 28 2024 |
| Licensing | Open Weight |
| Parameters | 14.3B |
| Context window | 32k |

## Tracked benchmark scores

| Benchmark | Score | Source | What it measures |
| --- | --- | --- | --- |
| HumanEval | 34.2% | Lab | 164 small Python problems: the AI is given a function's description and has to write the working function. This was the coding benchmark of the GPT-3.5 and GPT-4 era, before the field moved to fixing real bugs in real repositories. Higher is better. |
| MMLU | 62.5% | Lab | A 57-subject multiple-choice exam — history, law, medicine, maths — that was the standard measure of how much a model knows from 2020 until roughly 2024, when frontier scores crowded into the high 80s and labs moved on to harder tests. The scores here were published years apart under different testing setups, so read them as a historical record rather than a like-for-like ranking. Higher is better. |
| GSM8K | 61.5% | Lab | Grade-school maths word problems that take a few steps of arithmetic to work through. It separated the models of 2022 and 2023 sharply, then saturated. One caveat on the historical numbers: OpenAI included part of the GSM8K training set in GPT-4's pre-training mix, so GPT-4's score is not a clean few-shot result. Higher is better. |

## About Qwen1.5-MoE

Qwen1.5-MoE was Alibaba Cloud's first mixture-of-experts Qwen release. It arrived on March 28, 2024 with 14.3 billion total parameters and 2.7 billion active per token. The team built it by adapting the dense Qwen-1.8B checkpoint into a sparse model.

Qwen reported 62.5 on MMLU and 34.2 on HumanEval for the base checkpoint. The release tested whether a model could approach dense 7B performance with less computation per token, while the total weights still had to fit in memory.

## Questions and answers

### When was Qwen1.5-MoE released?

Qwen1.5-MoE was released by Qwen on Thursday, Mar 28 2024.

### Who made Qwen1.5-MoE?

Qwen1.5-MoE was built by Qwen. Alibaba's AI lab, building the Qwen family. The most prolific publisher of open-weight models of any major lab, alongside a closed Max and Plus tier sold through Alibaba Cloud.

### What benchmark scores did Qwen1.5-MoE get?

Qwen1.5-MoE reports 3 tracked benchmark scores — HumanEval: 34.2%; MMLU: 62.5%; GSM8K: 61.5%. Tracked scores may come from lab reports or independent benchmarks; source details accompany the benchmark data.

### What is the context window of Qwen1.5-MoE?

Qwen1.5-MoE has a context window of 32k. Providers may specify separate input and output token limits.

### How many parameters does Qwen1.5-MoE have?

Qwen1.5-MoE is reported at 14.3B parameters.

### Is Qwen1.5-MoE open source?

Partly. Qwen1.5-MoE is an open-weight model: the trained weights are free to download and run locally or on your own infrastructure, but the training data and code are not fully released and the license may restrict some commercial uses. It is not open source in the strict sense.

### What came before and after Qwen1.5-MoE?

Qwen's previous tracked release was Qwen1.5 on Feb 4 2024, 53 days earlier. It was followed by CodeQwen1.5 on Apr 16 2024.


---

Canonical page: https://aireleasetracker.com/model/qwen/qwen1.5-moe
Site index: https://aireleasetracker.com/llms.txt
Source: AI Release Tracker (https://aireleasetracker.com). Most benchmark scores come from lab launch material; gathered results identify the leaderboard that published them.
