Source: Deep Learning on Medium
Define-and-Run vs. Define-by-Run
Define-and-Run and Define-by-Run are frequently mentioned when introducing deep learning frameworks. These are the strategy to define and run neural network or computational graph. To understand and remember these concepts, I write down this post. Since this post is kind of summary, I hope you to read references to learn concepts deeply.
- Network(or computational graph) is defined and fixed.
- And then mini-batch data are fed into the predefined network.
- TensorFlow, Caffe, torch.nn, and Theano-based frameworks
- Network is defined dynamically via the actual forward computation.
- This dynamic definition allows conditionals and loops into the network definitions easily.
- Chainer, PyTorch