Key Specifications

Benchmark Performance

Benchmark	Score	Unit	Evaluated At	Notes	Source
MMLU	69.1	%	2024-07-16	5-shot	view
HUMANEVAL	49.9	pass@1	2024-07-16	—	view
GSM8K	72.4	%	2024-07-16	0-shot CoT	view
MATH	61	%	2024-07-16	0-shot CoT	view
BBH	69.8	%	2024-07-16	3-shot CoT	view
GPQA	37.5	%	2024-07-16	0-shot	view
IFEVAL	63.6	%	2024-07-16	prompt_strict	view
ARC	91.5	%	2024-07-16	challenge	view
MUSR	42.3	%	2024-07-16	0-shot	view
WINOGRANDE	71	%	2024-07-16	0-shot	view

Name: Mathstral 7B
Brand: mistral

Mistral Mathstral 7B 数学专用模型, 32K 上下文, 基于 Mistral 7B 微调, 在 MATH 基准上达到 56.6%。

Leverancier	Versie	Releasedatum	Contextvenster	Invoermodaliteiten	Uitvoermodaliteiten	Licentie
Mistral	mathstral-7b	2024-07-16	32K	text	text	Apache 2.0

Benchmark	Score	Eenheid	Notities
MMLU (Massive Multitask Language Understanding)	69.1	%	5-shot
HumanEval	49.9	pass@1	—
GSM8K (Grade School Math 8K)	72.4	%	0-shot CoT
MATH	61.0	%	0-shot CoT
BBH (BIG-Bench Hard)	69.8	%	3-shot CoT
GPQA	37.5	%	0-shot
IFEval	63.6	%	prompt_strict
ARC	91.5	%	challenge
MUSR	42.3	%	0-shot
WinoGrande	71.0	%	0-shot

Invoer	Uitvoer	Cache-lezen	Cache-schrijven
—	—	—	—

per miljoen tokens