AI Benchmarks, Explained: What MMLU and Friends Actually Measure
How to read a benchmark table without being fooled by it.
AI benchmarks for language models measure performance on curated test suites, but each score reflects a narrow, constructed view of capability. MMLU tests academic knowledge through multiple choice questions scored by token probabilities. Arena Elo captures pairwise human preference. HELM aggregates accuracy, fairness, and robustness across scenarios. Interpreting any benchmark number demands understanding how prompts, scoring rules, and contamination shape it.
A five point gain on MMLU can vanish if you change the prompt template by a few words. Yet leaderboards still treat that delta as a meaningful improvement. This article shows how the underlying machinery works, so you can read benchmark tables without being fooled by them.
Why benchmark scores are like exam grades, not IQ tests
Think of a benchmark as a specific exam that a student takes. A high grade means the student did well on that particular set of questions, using that particular grading scheme. It does not mean the student is generally smart, creative, or capable in all domains. One exam might reward memorization. Another might reward clever problem solving under time pressure. Two students with the same grade on different exams may have entirely different strengths.
Benchmark scores work the same way. MMLU is a multiple choice exam spanning 57 subjects, scored by picking the option the model assigns the highest probability. A model scoring 87% did not “know” 87% of all human knowledge. It performed well on those 57 domains, using that question format, under a specific prompt template, with no real world interaction. Swap the exam for Chatbot Arena, and the same model gets an Elo rating that depends entirely on which other models were in the pool and which users wrote the prompts. Neither number is a complete picture.
The rest of this article unpacks how the most cited benchmarks actually work, where they mislead, and how to read scores the way an engineer who designs evaluation pipelines would.
How does MMLU actually decide which answer is “correct”?
The core mechanic is a log likelihood selection over answer option tokens. The evaluator constructs a prompt that ends with the question and a cue like “Answer:”. It then appends each candidate answer token (usually “A”, “B”, “C”, or “D”) and asks the model for the probability of that token given the preceding context. The option with the highest probability wins. This is deterministic and avoids sampling noise.
For a question (q) with options labeled (a_i), the score for (a_i) is (P(a_i \mid q)) as computed by the model’s output logits for that token position. The evaluation harness uses the canonical prompt format defined by EleutherAI’s lm-evaluation-harness, which standardizes how questions are wrapped and how few-shot examples are inserted. If you change the prompt template, you change the distribution over tokens and therefore the score.
Because only a single token is scored, tokenization quirks matter. Some models tokenize “A” differently than others. The harness normalizes for this, but tiny implementation differences still cause score drift between lab and production. That is why the first thing any reproduction effort should check is the exact prompt template and tokenizer configuration.
Why do small prompt changes nudge MMLU scores by 4-5%?
A model’s probability distribution over “A”, “B”, “C”, “D” is sensitive to the surrounding words. Replacing “Answer:” with “Solution:” can shift those probabilities enough to flip several percent of answers. The model is not just retrieving knowledge. It is also modelling how likely a particular continuation seems given the style of the prompt.
MMLU-Pro reduced this sensitivity by expanding each question to ten answer options and by filtering out noisy items. In experiments, prompt-variant score swings shrank from about 4-5% on original MMLU to around 2% on MMLU-Pro. If you see two models with a 3-point gap on MMLU-Pro, you can be more confident it reflects a real difference. The same gap on original MMLU could easily be a prompt artifact.
MMLU-Pro also increases difficulty dramatically. With ten options, random guessing yields 10% accuracy instead of 25%. Models lose 16 to 33 percentage points when moving from MMLU to MMLU-Pro. That drop is itself a useful signal. The larger the drop, the more the original score relied on superficial recognition rather than robust reasoning.
When comparing models, always check which benchmark variant is being reported and whether prompt-sensitivity numbers are disclosed. A difference that looks significant may be invisible once you switch to MMLU-Pro or even a slightly different template.
What does a Chatbot Arena Elo rating actually represent?
Arena Elo is a relative rating from pairwise human preference votes. A user sees two anonymized model outputs for a prompt and picks the one they prefer. The system treats each human choice as a game outcome and updates ratings with the Elo algorithm, just like chess ratings.
A rating of 1300 does not mean “the model is 1300 good at coding.” It means that, within the pool of models being compared and under the distribution of prompts submitted by Arena users, this model wins more pairwise comparisons than models with lower ratings. If the pool changes, if five new stronger models join, all existing ratings shift downward. An Elo number is only interpretable within its snapshot of the pool.
Style, tone, conciseness, and refusal behavior strongly influence votes. A model can have a high Elo because it writes friendly responses, not because it answers facts correctly. There is no ground-truth correctness check. That makes Arena Elo a useful measure of user experience but a terrible measure of factual accuracy or task-specific capability. Never treat it as a substitute for accuracy-oriented benchmarks when factual reliability matters.
How does contamination inflate benchmark scores?
Contamination happens when test questions appear in the model’s training data. If a model has memorized MMLU answers, it can regurgitate the correct option without reasoning. This produces scores that look like real understanding but crumble when questions are rephrased or when a contamination-free variant is used.
MMLU-CF addresses this by curating questions with strict decontamination rules and keeping the test set closed-source. The test set is not publicly downloadable. Evaluators must submit model outputs to a service that returns scores. When the same models that scored well on open MMLU are tested on MMLU-CF, gaps appear. Some models that reproduced exact answer choices on the original test set fell back to near-random guessing on the decontaminated version.
If you need a trustworthy knowledge score, look for benchmarks that explicitly describe decontamination steps and keep test data private. Alternatively, run your own evaluation on a privately held holdout set that you know never appeared in public training corpora. Trusting a published MMLU score without contamination analysis is gambling that the model never saw the test set.
Why does HELM report seven metrics instead of one number?
HELM, the Holistic Evaluation of Language Models, runs models through dozens of scenarios and computes accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency. A single accuracy number can hide that a model answers correctly most of the time but produces toxic output on 5% of prompts, or that its calibration is so poor the confidence scores are useless.
The spider plots in HELM show trade-offs explicitly. A model might have higher accuracy than another but worse fairness. A human reading the full report can decide which trade-offs matter for their use case. If you only look at the top row of a leaderboard, you miss risks that will matter in production.
For an engineering team deploying a model in a customer facing product, HELM’s multi-metric approach is far more actionable than a single MMLU number. The cost is complexity. You need to set aside time to read the scenario-by-scenario breakdowns and decide which dimensions are constraints and which are nice-to-have.
What does SWE-bench actually test that snippet benchmarks miss?
SWE-bench gives a model a full GitHub repository and a real issue description, then asks for a patch. Success is measured by running the repository’s existing test suite. This is end-to-end software engineering, not isolated function completion. The model must navigate real directory structures, understand project conventions, and produce changes that pass tests across multiple files.
Traditional code benchmarks present a function signature and a docstring, then score the output token-by-token. Those benchmarks reward surface-level pattern matching. SWE-bench rewards the ability to debug, make architectural decisions, and handle edge cases that only surface when tests execute. The pass rate metric is often pass@k, meaning at least one of k generated patches passes all tests, which acknowledges that stochastic generation may need multiple attempts.
- Function signature and docstring
- Token-level output scoring
- Tests surface pattern matching
- Full repository and issue description
- Patch must pass existing test suite
- Tests engineering capability
SWE-bench scores are lower and gaps between models are wider than on snippet benchmarks. That makes it a stronger signal for engineering capability, but also more expensive to run. Setting up reproducible evaluation environments for thousands of repositories is nontrivial. When you see a SWE-bench score, always check which version of the benchmark was used and whether the evaluation harness was reproduced faithfully.
Quick Reference
| Benchmark | Measures | Metric | Watch out for |
|---|---|---|---|
| MMLU | Academic multitask knowledge | Accuracy (log-likelihood selection) | Prompt sensitivity, contamination, saturation |
| MMLU-Pro | Reasoning-heavy multitask with 10 options | Accuracy | Still contamination-vulnerable if open test set |
| MMLU-CF | Multitask knowledge with strict decontamination | Accuracy | Closed test set limits independent reproduction |
| Arena Elo | Human pairwise preference | Elo rating | Relative to model pool, style heavily influences scores |
| HELM | Holistic scenario-by-scenario analysis | Multi-metric spider plots | Overinterpretation of any single sub-metric, requires reading full report |
| SWE-bench | Real-world software patch success | Pass@k (passing tests) | Environment setup cost, non-trivial execution overhead |
Frequently Asked Questions
Q: Why does my local MMLU evaluation differ from the published numbers? Even small differences in prompt template, few-shot examples, or tokenizer configuration shift scores by several percent. To get comparable numbers, use exactly the same evaluation harness (lm-evaluation-harness) and the same template. Otherwise, treat your numbers as a separate measurement.
Q: How can I tell if a model's benchmark score is inflated by contamination? Check whether the model reproduces original answer options verbatim when prompted with only the question stem. Then test it on a contamination-free variant like MMLU-CF or on paraphrased versions of the questions. A sharp drop on those variants is a strong signal of memorization, not generalization.
Q: Is Arena Elo the best measure of human preference? It is a reasonable crowd-sourced signal of stylistic preference under a specific user base. It is not absolute and does not guarantee task utility. For applications where factual correctness or safety is critical, complement Elo with task-specific accuracy benchmarks.
Q: Why do some benchmarks report accuracy and others pass@k? Accuracy is natural for single-selection tasks where the model must get the answer right in one attempt. Pass@k is used in generative coding benchmarks because models often need several attempts to produce a patch that passes all tests. It rewards a model that can eventually get it right, which is closer to how developers work.
Q: When should I choose HELM over a single benchmark like MMLU? Use HELM when you need a multi-dimensional view of correctness, calibration, fairness, and toxicity, and you can invest time interpreting full scenario reports. For quick sanity checks or when only knowledge accuracy matters, a single benchmark suffices but may hide risks that surface later.
If you want this kind of breakdown every week, how real systems actually work under the hood, subscribe to Internals Decoded at internalsdecoded.com.
Sources
- Chatbot Arena paper
- MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
- BIG-bench: Beyond the Imitation Game
- HELM: Holistic Evaluation of Language Models
- MMLU: Measuring Massive Multitask Language Understanding
- BIG-Bench-Hard
- MMLU-CF: A Contamination-Free Multitask Language Understanding Benchmark
- AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
- EleutherAI lm-evaluation-harness
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark
- HellaSwag: Can a Machine Really Finish Your Sentence?
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- MT-Bench-101: A Fine-Grained Benchmark for Multi-Turn Dialogue Capabilities