Attention Is Not Explanation

Authors: Jain & Wallace Year: 2019 ArXiv/Link: https://arxiv.org/abs/1902.10186

Summary

Critiques the assumption that attention weights explain model decisions, highlighting limitations of attention visualization.

Key Concepts

• Attention critique
• Interpretability limits
• Explanation assumptions
• Attention patterns
• Model transparency

Impact

Cautions against over-interpreting attention weights

Category

Interpretability

---