Define-and-Run vs. Define-by-Run

Source: Deep Learning on Medium

Define-and-Run vs. Define-by-Run

Define-and-Run and Define-by-Run are frequently mentioned when introducing deep learning frameworks. These are the strategy to define and run neural network or computational graph. To understand and remember these concepts, I write down this post. Since this post is kind of summary, I hope you to read references to learn concepts deeply.



  • Network(or computational graph) is defined and fixed.
  • And then mini-batch data are fed into the predefined network.
  • TensorFlow, Caffe, torch.nn, and Theano-based frameworks


  • Network is defined dynamically via the actual forward computation.
  • This dynamic definition allows conditionals and loops into the network definitions easily.
  • Chainer, PyTorch