BLK · QUALITY LEADERBOARDSreviewed · per-quant · raw logs

Quality benchmarks

Reviewed quality benchmark scores for open-weight local models. First-party rows are produced on RunLocalAI hardware; community rows render only after review. Each public score carries a stated quantization, runtime, hardware target, and raw-log Gist.

Benchmarks tracked
3
Total test runs
0
Unique models
0
First-party
0
+Submit a scoreMethodology →JSON API →·Inference speed leaderboard (tok/s) →
BENCHMARK · HUMANEVAL-PLUS

HumanEval+ (EvalPlus)

164 Python coding problems with augmented test suites (~80 hidden tests per problem) from the EvalPlus extension of the original HumanEval benchmark. Measures functional code gener

0 test runs
Coding
No test runs yet for HumanEval+ (EvalPlus). Runs land here as our weekend compute completes. If you have rig time and want to contribute, see /benchmarks/submit.
Full HumanEval+ (EvalPlus) leaderboard →
BENCHMARK · MBPP-PLUS

MBPP+ (EvalPlus)

EvalPlus's augmented MBPP suite: introductory Python programming tasks with stronger hidden tests than the original MBPP benchmark. Measures short-form functional code generation q

0 test runs
Coding
No test runs yet for MBPP+ (EvalPlus). Runs land here as our weekend compute completes. If you have rig time and want to contribute, see /benchmarks/submit.
Full MBPP+ (EvalPlus) leaderboard →
BENCHMARK · TURKISH-MMLU-GENERATIVE

TurkishMMLU (Generative)

Turkish-translated Massive Multitask Language Understanding benchmark: 900 questions across 9 subjects (Biology, Chemistry, Geography, History, Mathematics, Philosophy, Physics, Re

0 test runs
Turkish
General knowledge
Multilingual
No test runs yet for TurkishMMLU (Generative). Runs land here as our weekend compute completes. If you have rig time and want to contribute, see /benchmarks/submit.
Full TurkishMMLU (Generative) leaderboard →
HOW SCORES EARN PUBLIC RENDER

The trust gate

  • Every row must link to a public Gist with the raw stdout + stderr of the run. No Gist, no public render.
  • Trust tier shown per row: first-party (we ran it), verified (community submission reviewed with named verifier, timestamp, raw log, and reproduction command), pending (submitted but awaiting review and hidden from public leaderboards).
  • Reproduction commands published so a third party can replicate independently. Full methodology lives at /benchmarks/methodology.
  • Same model + quant + runtime + hardware tuple is unique. Anonymous submissions cannot overwrite an existing public row; reviewers decide whether a new run replaces or supplements the record.