Skip to content

Playing Atari with Deep Reinforcement Learning (Deep Q-Networks)

Authors: Volodymyr Mnih, Koray Kavukcuoglu David Silver, Alex Graves, Ioannis Antonoglou, et al. Year: 2013 Venue: arXiv 2013, NIPS 2013 Workshop Citations: 30,000+

Summary

Introduces Deep Q-Networks (DQN), combining Q-learning with deep neural networks for Atari game playing. Uses experience replay and target network for stable learning. First deep RL algorithm achieving human-level performance on complex tasks.

Key Concepts

  • Q-Learning: Off-policy temporal difference learning algorithm
  • Value Function: Learns state-action value (Q) function
  • Experience Replay: Stores transitions and replays for training
  • Target Network: Separate network reduces correlation in targets
  • Epsilon-Greedy: Balances exploration and exploitation
  • Bellman Equation: Iteratively improves value estimates

Impact

  • 30,000+ citations
  • Sparked deep reinforcement learning revolution
  • Showed deep learning applicable to control problems
  • Inspired DQN variants (Double DQN, Dueling DQN, Rainbow)
  • Foundation for many RL algorithms
  • Demonstrated AI can learn complex behaviors from pixels
  • Influenced modern game-playing AI systems

Key Results

  • Achieved human-level performance on Atari games
  • Single architecture worked across diverse games
  • Better generalization than previous RL methods
  • Trained end-to-end from raw pixels
  • Double DQN (van Hasselt et al., 2016)
  • Dueling DQN (Wang et al., 2015)
  • Rainbow DQN (Hessel et al., 2017)
  • AlphaGo (Silver et al., 2016)