Skip to content

Densely Connected Convolutional Networks (DenseNet)

Authors: Gao Huang, Zhuang Liu, Laurens van der Maaten, Kilian Q. Weinberger Year: 2017 (arXiv 2016) Venue: CVPR 2017 Citations: 20,000+

Summary

Introduces DenseNet, connecting each layer to all previous layers in feed-forward fashion. Dense connections enable feature reuse, reduce vanishing gradient, encourage feature propagation. Achieves state-of-the-art ImageNet performance.

Key Concepts

  • Dense Connections: Each layer receives input from all preceding layers
  • Feature Reuse: Earlier features used multiple times, reducing redundancy
  • Implicit Deep Supervision: Dense connections provide implicit supervision
  • Reduced Feature Map Size: Dense connections reduce channels needed
  • Computational Efficiency: Fewer parameters than comparable ResNets
  • Regularization Effect: Dense connections act as regularizer

Impact

  • 20,000+ citations
  • Strong alternative to ResNet and other architectures
  • Better parameter efficiency than ResNet
  • Improved generalization on transfer learning
  • Influenced subsequent dense connection research
  • Foundation for understanding feature reuse
  • Still widely used in modern vision models

Key Results

  • State-of-the-art ImageNet classification
  • Better parameter efficiency than ResNet
  • Improved generalization to new datasets
  • Faster training convergence
  • ResNet: Deep Residual Learning (He et al., 2015)
  • Highway Networks (Srivastava et al., 2015)
  • Inception Networks (Szegedy et al., 2014)
  • EfficientNet (Tan & Le, 2019)