Skip to content

Squeeze-and-Excitation Networks

Authors: Jie Hu, Li Shen, Samuel Albanie, Gang Sun, Enhua Wu Year: 2018 (arXiv 2017) Venue: CVPR 2018 Citations: 15,000+

Summary

Introduces Squeeze-and-Excitation (SE) blocks, adaptive channel-wise feature recalibration. Uses global average pooling (squeeze) and fully connected layers (excitation) to weight channels. Simple, effective module improving ResNet and other architectures.

Key Concepts

  • Channel Attention: Learns importance of each feature map
  • Squeeze Operation: Global average pooling across spatial dimensions
  • Excitation Operation: FC layers predict channel importance weights
  • Lightweight: Minimal computational overhead
  • Plug-and-Play: Easily adds to existing architectures
  • Adaptive Feature Recalibration: Dynamically weights channels

Impact

  • 15,000+ citations
  • Simple attention mechanism for CNNs
  • Widely adopted in image classification models
  • Influenced subsequent attention mechanisms
  • Easy to implement and integrate
  • Consistent improvements across architectures
  • Foundation for channel attention research

Key Results

  • Improved ResNet and other base architectures
  • Minimal computational cost for accuracy gain
  • Better transfer learning performance
  • Applicable across various domains
  • Spatial Attention (Concurrent work)
  • CBAM: Convolutional Block Attention (Woo et al., 2018)
  • Attention Mechanisms (Vaswani et al., 2017)
  • ResNet (He et al., 2015)