Benchmarking & Evaluation (2004-2023)¶
Benchmark datasets and evaluation frameworks for measuring model performance.
NLU Benchmarks¶
- GLUE: A Multi-Task Benchmark for NLU - NLU benchmarks
- SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems - Harder NLU tasks
Knowledge & Instruction Following¶
- MMLU: Measuring Massive Multitask Language Understanding - Knowledge benchmark
- MT-Bench: A Benchmark for Evaluating Language Model Instruction Following - Instruction following
Evaluation Metrics¶
- BERTScore: Evaluating Text Generation with BERT - Semantic similarity metric
- ROUGE: A Package for Automatic Evaluation of Summarization - Summarization metrics
Key Insight: Good benchmarks are essential for objective evaluation and tracking progress in AI research.