1.LSTM&GRU的原理
https://blog.csdn.net/jerr__y/article/details/58598296
https://github.com/starflyyy/Gated-Recurrent-Unit-GRU
2.多层LSTM
pytorch里有一个num_layers,是指参数共享之后网络也有不同cell，即相当于隐含层的数目，是指cell串联和mlp很像，即为StackedRNN具体如下图
3.双向RNN
待继续了解https://blog.csdn.net/jojozhangju/article/details/51982254
4.lstm&rnn的实现和改造
源码分析https://zhuanlan.zhihu.com/p/63638656
例子分析https://zhuanlan.zhihu.com/p/32103001，可以加上自己改写RNNcellhttps://github.com/huyingxi/new-LSTM-Cell
pytorch中rnn的输出有output和hidden，区别如下图（lstm）
其中：
Outputs: output, (h_n, c_n)
- output (seq_len, batch, hidden_size * num_directions): tensor containing the output features (h_t) from the last layer of the RNN, for each t. If a torch.nn.utils.rnn.PackedSequence has been given as the input, the output will also be a packed sequence.
- h_n (num_layers * num_directions, batch, hidden_size): tensor containing the hidden state for t=seq_len
- c_n (num_layer, batch, hidden_size): tensor containing the cell state for t=seq_len
一种实现，但其实现的多层lstm不太对https://github.com/emadRad/lstm-gru-pytorch/