Source: Deep Learning on Medium
What is RNN ?
In RNN, output is produced by combination of current inputs and past inputs. Differ from feedforward, it uses the output as next step’s input. It stores past input data that has a meaning for the output. Output that comes from hidden layer is written to units named as content units. So, feedforward is not enough for this type of storing. Following equation is used for calculating:
→ Where ht is result of hidden layer at moment t, Wxt is dot product and Uh(t-1) is dot product of weight U and h(t-1) which is stored in content unit. Addition is given to activation function like sigmoid or tanh.