MMLU's 57-subject knowledge test used to be the first number every model release led with. Then frontier models started hitting 90% on it consistently, and state-of-the-art models clustered within 2-4% accuracy, limiting the benchmark's ability to differentiate incremental advances. A number that can't tell GPT-5.3 from Claude Opus is not measuring much. That's the evals problem in a sentence.
But evals aren't one thing. The word covers three distinct methods-code-based checks, human annotation, and LLM-as-judge-each with different mechanics and different ways of going wrong. Understanding which one produced a score tells you more than the score itself.
How deterministic evals work (and where they shine)
A deterministic eval has a single correct answer. The model either gets it or doesn't. A user query like "can I see the apartment on July 4th, 2026?" has one correct interpretation-the AI's job is to extract that date and put it into the right format for a downstream tool-and this is a deterministic, objective task that's either right or wrong.
The strongest example in code generation is HumanEval. The dataset consists of 164 programming challenges; each problem includes a function signature, docstring, body, and several unit tests, with an average of 7.7 tests per problem. The model generates code, the code runs, and the unit tests either pass or they don't. No rubric, no judgment.
The metric used is pass@k: the probability that at least one of the top k-generated code samples for a problem passes the unit tests.
Pass@1 measures the probability that a single generated solution is correct; Pass@10 and Pass@100 measure the probability that at least one solution among 10 or 100 attempts is correct. For an inline autocomplete tool, pass@1 is what matters. For a background agent generating options for a human to review, pass@10 is more honest.
The limitation of deterministic evals is scope. They work cleanly for tasks with objective answers: code that runs, dates parsed correctly, JSON that validates. They can't touch tone, reasoning quality, or anything where "correct" is contextual.
Beagle in action#engineering, weekly model reviewHow human annotation evals work (and why they're expensive)
When the task can't be reduced to pass/fail, someone has to read the output and judge it. Spending 30 minutes manually reviewing 20-50 LLM outputs whenever you make significant changes, using one domain expert who understands your users as the quality decision-maker, is a legitimate baseline strategy.
Model evaluation involves computing performance using a task, dataset, and metric. For human annotation, the metric is usually inter-annotator agreement-how often two independent humans give the same rating. When that agreement is high, you trust the signal. When it's low, your rubric is ambiguous, not your model.
The cost is real. Benchmark-judge scoring with an LLM costs roughly $77, compared with an estimated $5,000-$7,500 for human annotation-a reduction of 98-99%. That gap is why teams reach for LLM-as-judge. But it's worth noting that human annotation, done well, remains the ground truth the other methods try to approximate.
Code-based evals are the perfect tool for simple failures. To build one, teams first assemble a "golden dataset" of test cases, brainstorming the many different ways users might phrase something, focusing on common patterns and tricky edge cases-the goal is a comprehensive test suite that can reliably catch regressions. That golden dataset is the thing worth building. Every task type eventually benefits from one specific to your domain.
How LLM-as-judge evals work (and the bias hiding inside them)
The LLM-as-a-judge paradigm uses large language models as automated evaluators for open-ended tasks, offering a scalable alternative to human evaluation. In practice: you show a judge model one or two outputs, ask it to rate quality on a rubric or pick the better response, and aggregate scores across many examples.
Studies show LLMs achieve high agreement with human experts on many tasks, which is why this approach has taken off. The cost-to-signal ratio is hard to beat. But three systematic biases undermine the scores if you don't account for them.
Verbosity bias. Models systematically assign higher scores to longer outputs even when additional content provides no substantive improvement.
Position bias. Models show significant preference for responses presented in specific positions regardless of content quality. If you always put the challenger model's response second, the incumbent gets a structural advantage.
Self-preference bias. LLM-based evaluation frameworks may introduce self-preference bias, where models disproportionately favor their own generated responses over those produced by humans or alternative models. This one is the most structurally dangerous. On ArenaHard, self-preferential bias ranges from -38% to +90%-some models over-rate their own answers relative to other judges, others under-rate them. A swing of 128 percentage points on a single dataset is not noise.
Empirical analysis across 20 mainstream LLMs reveals that advanced capabilities are often uncorrelated, or even negatively correlated, with low self-preference bias. Being good at generation does not make a model a fair judge.
The practical fix: position bias can be addressed by rearranging the order of comparative options within the prompt. Verbosity bias is harder-length-controlled metrics like those in AlpacaEval use regression-based debiasing. Self-preference bias is best avoided by using a different model family as your judge than the one you're evaluating.
The contamination problem no benchmark card mentions
A key challenge in benchmark aggregation is saturation: once a critical mass of models exceeds 90% accuracy on a benchmark, it ceases to discriminate among frontier systems. MMLU, for example, was the dominant general benchmark through 2023 but saturated as frontier models consistently exceeded 90% accuracy.
Saturation is bad enough. Contamination is worse, because it inflates scores before saturation sets in. Many widely-used benchmarks are highly contaminated: when checked against the DCLM-baseline training corpus, MMLU has 27.70% dirty entries, MMLU-Pro has 16.20%, ARC-Challenge has 32.6%, and ARC-Easy has 32.3%.
The mechanism: some LLMs, due to their memorization capabilities, directly provide choices identical to the original MMLU question when those questions are used as prompts. The model isn't reasoning-it's recalling. The result is performance inflation: reported scores systematically overstate genuine capability, and the apparent rate of progress on the benchmark diverges from actual progress on the underlying skill.
The 13% GSM8K accuracy drop observed when contaminated examples were removed is a documented lower bound, not an upper bound. The true effect is larger and harder to measure because training datasets aren't fully disclosed.
Build your eval before you pick your model
Evaluation is part of the development process rather than a distinct line item, similar to how debugging is part of software development-you should always be doing error analysis.
That means a few practical things:
Match the eval type to the task. Structured outputs get code-based evals. Open-ended text gets human annotation or LLM-as-judge with explicit bias controls.
Build a golden dataset specific to your use case. A model's published benchmark score predicts production performance only when the benchmark tests tasks similar to your use case, the test set is clean of training data contamination, and the benchmark hasn't saturated to the point where score differences are statistically meaningless.
Rotate your judge model. Never use the same model family as both the subject and the judge.
Track regressions over time. A score taken once during model selection is not an ongoing eval. Run your suite on every new model version.
A teammate like Beagle, sitting inside Slack where your team's actual decisions get made, can pull the latest eval scores from your tracking sheet and post them in context-so the discussion of which model to use is grounded in your numbers, not a vendor's.
Beagle in action#ai-infra, Monday standupHow LLM evals work: common questions
What is LLM-as-judge and how does it work?
LLM-as-judge uses a large language model to rate or compare outputs from another model, replacing expensive human annotation. The judge receives a prompt, one or two outputs, and a rubric, then returns a score or preference. It's fast and cheap but prone to position bias, verbosity bias, and self-preference bias, which require explicit mitigation.
What is the pass@k metric in LLM evaluation?
Pass@k measures the probability that at least one of k generated code samples for a problem passes all unit tests. Pass@1 reflects single-attempt accuracy; Pass@10 reflects whether any of ten attempts is correct. It's a deterministic metric-no human judgment required-which makes it reliable for code and structured-output tasks.
Why are MMLU benchmark scores unreliable?
Two reasons: saturation and contamination. Frontier models now cluster within 2-4% on MMLU, making differences statistically meaningless. Separately, research shows 27.70% of MMLU entries appear in common training corpora, meaning some scores reflect memorization rather than capability. Use newer benchmarks like MMLU-Pro or task-specific golden datasets instead.
What is benchmark contamination in AI?
Benchmark contamination happens when a model's training data includes questions from the evaluation set, so the model effectively has the answers memorized. When contaminated examples are removed from GSM8K, accuracy drops by 13%-and that's a lower bound, since full training data disclosure is rare.
How should a team build their own LLM evals?
Start with a golden dataset: 50-100 real examples from your actual task, with known-good outputs as reference answers. Define a small rubric (3-5 dimensions). Use code-based pass/fail where possible; use LLM-as-judge with a different model family for open-ended quality. Run the eval on every candidate model and on every significant prompt change.