MMLU: Measuring Massive Multitask Language Understanding

Authors: Hendrycks et al. Year: 2021 ArXiv/Link: https://arxiv.org/abs/2009.03300

Summary

Large-scale benchmark of 57 tasks across STEM, social sciences, humanities testing broad knowledge and reasoning.

Key Concepts

• Knowledge benchmark
• Multitask learning
• Broad evaluation
• Reasoning assessment
• Domain diversity

Impact

Widely-used benchmark for evaluating model knowledge

Category

Benchmark Papers

---