Compare AI models

Specifications
Parameters
8B6B
Context window
1M8k
Benchmarks
MATH
35.9%25.7%
MMLU
68.1%61.4%
Overview
CompanyGoogleZ.ai
Release dateOct 3 2024Oct 27 2023
AccessClosedOpen Weight
Model detailsView modelView model

Frequently asked questions

Gemini 1.5 Flash-8B leads ChatGLM3-6B on 2 of the 2 benchmarks they both report (MATH, MMLU). ChatGLM3-6B shipped 342 days before Gemini 1.5 Flash-8B, so benchmark comparisons should account for the intervening progress.

Gemini 1.5 Flash-8B has 8B parameters, while ChatGLM3-6B has 6B. Context windows are 1M (Gemini 1.5 Flash-8B) vs 8k (ChatGLM3-6B). Gemini 1.5 Flash-8B is closed, while ChatGLM3-6B is open weight.

On MATH, Gemini 1.5 Flash-8B leads at 35.9% vs ChatGLM3-6B at 25.7%. On MMLU, Gemini 1.5 Flash-8B leads at 68.1% vs ChatGLM3-6B at 61.4%.