Accuracy Hides How Language Models Fail: Measuring Failure States Under Matched Output Budgets

Language-model benchmarks collapse two distinct measurement questions into a single accuracy score: whether a response reached an evaluable state, and whether its answer was judged correct. We introduce a two-layer evaluation framework that separates scorer-independent execution evidence, including ...