Pytorch 教程 LSTM

Question

我试图用 Pytorch 实现关于序列模型和长短期记忆网络的the exercise。这个想法是添加一个 LSTM 词性标注器字符级功能，但我似乎无法解决。他们暗示应该涉及两个 LSTM，一个将输出字符级表示，另一个将负责预测词性标签。我只是不知道如何循环遍历单词级别（在句子中）和字符（在句子的每个单词中）并在 forward 函数中实现它。有谁知道该怎么做？还是遇到类似情况？

这是我的代码：

class LSTMTaggerAug(nn.Module):
def __init__(self, embedding_dim_words, embedding_dim_chars, hidden_dim_words, hidden_dim_chars, vocab_size, tagset_size, charset_size):
    super(LSTMTaggerAug, self).__init__()
    self.hidden_dim_words = hidden_dim_words
    self.hidden_dim_chars = hidden_dim_chars
    self.word_embeddings = nn.Embedding(vocab_size, embedding_dim_words)
    self.char_embeddings = nn.Embedding(charset_size, embedding_dim_chars)
    self.lstm_char = nn.LSTM(embedding_dim_chars, hidden_dim_chars)
    self.lstm_words = nn.LSTM(embedding_dim_words + hidden_dim_chars, hidden_dim_words)
    self.hidden2tag = nn.Linear(hidden_dim_words, tagset_size)
    self.hidden_char = self.init_hidden(c=False)
    self.hidden_words = self.init_hidden(c=True)

def init_hidden(self, c=True):
    if c:
        return (autograd.Variable(torch.zeros(1, 1, self.hidden_dim_words)),
                autograd.Variable(torch.zeros(1, 1, self.hidden_dim_words)))
    else:
        return (autograd.Variable(torch.zeros(1, 1, self.hidden_dim_chars)),
                autograd.Variable(torch.zeros(1, 1, self.hidden_dim_chars)))


def forward(self, sentence, words):
    # embeds = self.word_embeddings(sentence)
    for ix, word in enumerate(sentence):
        chars = words[ix]
        char_embeds = self.char_embeddings(chars)
        lstm_char_out, self.hidden_char = self.lstm_char(
            char_embeds.view(len(chars), 1, -1), self.hidden_char)
        char_rep = lstm_char_out[-1]
        embeds = self.word_embeddings(word)
        embeds_cat = torch.cat((embeds, char_rep), dim=1)
        lstm_out, self.hidden_words = self.lstm_words(embeds_cat, self.hidden_words)
        tag_space = self.hidden2tag(lstm_out.view(1, -1))
        tag_score = F.log_softmax(tag_space, dim=1)
        if ix==0:
            tag_scores = tag_score
        else:
            tag_scores = torch.cat((tag_scores, tag_score), 0)

    return tag_scores

Answer 1

根据您的描述，最幼稚的做法是使用去掉标点符号的句子s。然后拆分成单词：

words = s.split()

获取您的第一个字符级 lstm LSTMc 并将其分别应用于每个单词以对单词进行编码（使用 lstm 的最后输出状态对单词进行编码）：

encoded_words = []
for word in words:
    state = state_0

    for char in word:
        h, state = LSTMc(one_hot_encoding(char), state)
    encoded_words.append(h)

在对单词进行编码后，将词性标注器 lstm LSTMw 传递给已编码的单词：

state = statew_0
parts_of_speech = []
for enc_word in encoded_words:
    pos, state = LSTMw(enc_word, state)
    parts_of_speech.append(pos)