Key Specifications

Benchmark Performance

Benchmark	Score	Unit	Evaluated At	Notes	Source
MMLU	60	%	2023-08-17	5-shot	view
HUMANEVAL	79.8	pass@1	2023-08-17	—	view
GSM8K	46.7	%	2023-08-17	0-shot CoT	view
MATH	40.6	%	2023-08-17	0-shot CoT	view
BBH	67.2	%	2023-08-17	3-shot CoT	view
GPQA	36.6	%	2023-08-17	0-shot	view
IFEVAL	53.8	%	2023-08-17	prompt_strict	view
ARC	91.2	%	2023-08-17	challenge	view
MUSR	53.1	%	2023-08-17	0-shot	view
WINOGRANDE	70.4	%	2023-08-17	0-shot	view

Name: Microsoft WizardCoder Python 34B
Brand: other

Microsoft WizardCoder Python 34B 代码模型, 16K 上下文, 基于 Code Llama 34B, Python 代码生成突出。

Fornitore	Versione	Data di rilascio	Finestra di contesto	Modalità di input	Modalità di output	Licenza
Other	wizardcoder-python-34b	2023-08-17	16K	text	text	Llama 2 Community License

Benchmark	Punteggio	Unità	Note
MMLU (Massive Multitask Language Understanding)	60.0	%	5-shot
HumanEval	79.8	pass@1	—
GSM8K (Grade School Math 8K)	46.7	%	0-shot CoT
MATH	40.6	%	0-shot CoT
BBH (BIG-Bench Hard)	67.2	%	3-shot CoT
GPQA	36.6	%	0-shot
IFEval	53.8	%	prompt_strict
ARC	91.2	%	challenge
MUSR	53.1	%	0-shot
WinoGrande	70.4	%	0-shot

Input	Output	Lettura cache	Scrittura cache
—	—	—	—

per milione di token