如何将 KERAS `MLP` 模型转换为 `Pythorch` 模型

Question

我有一个使用某些项目的MLP 模型

from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers import Embedding

model = Sequential()
embedding_layer = Embedding(vocab_size, 50, input_length=len(X[0]))
model.add(embedding_layer)
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dense(3, activation='softmax'))
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()
model.fit(X, Y, epochs=10, verbose=1)

但我想将此MLP 模型转换为Pythorch 模型。

我该怎么做？

非常感谢。

我添加此代码以生成data：

def generate_batch_data(x, y, batch_size):
    i, batch = 0, 0
    for batch, i in enumerate(range(0, len(x) - batch_size, batch_size), 1):
        x_batch = x[i : i + batch_size]
        y_batch = y[i : i + batch_size]
        yield x_batch, y_batch
    if i + batch_size < len(x):
        yield x[i + batch_size :], y[i + batch_size :]
    if batch == 0:
        yield x, y

还有这段代码：

epochs = 10
batch_size = 10
for epoch in range(epochs):
    print(f'Epoch {epoch+1}/{epochs}')
   
    for x_train, y_train in generate_batch_data(x_train, y_train, batch_size):
        y_hat = model(x_train)
        loss = criterion(y_hat, y_train)
        acc = (y_hat.argmax(1) == y).float().mean()

        print(f'loss: {loss}, accuracy: {acc}')

这次我犯了这个错误：

RuntimeError                              Traceback (most recent call last)
<ipython-input-50-7ffa61cb7f34> in <module>()
      5 
      6     for x_train, y_train in generate_batch_data(x_train, y_train, batch_size):
----> 7         y_hat = model(x_train)
      8         loss = criterion(y_hat, y_train)
      9         acc = (y_hat.argmax(1) == y).float().mean()

4 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1850         # remove once script supports set_grad_enabled
   1851         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1852     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1853 
   1854 

RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.FloatTensor instead (while checking arguments for embedding)  

我在这里添加了torch.LongTensor：

def tokenize_and_pad_text(df, max_seq):
  tokenized_text = tokenize_text(df, max_seq)
  padded_text = pad_text(tokenized_text, max_seq)
  return torch.LongTensor(padded_text)

train_indices = tokenize_and_pad_text(df_train, max_seq)

x_train = bert_model(train_indices)[0]

并在此处更改：

y_hat = model(x_train.long())

但这一次它给出了这个错误：

IndexError                                Traceback (most recent call last)
<ipython-input-67-9ad38a8c062a> in <module>()
      5 
      6     for x_train, y_train in generate_batch_data(x_train, y_train, batch_size):
----> 7         y_hat = model(x_train.long())
      8         loss = criterion(y_hat, y_train)
      9         acc = (y_hat.argmax(1) == y).float().mean()

4 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1850         # remove once script supports set_grad_enabled
   1851         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1852     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1853 
   1854 

IndexError: index out of range in self

模型是这样的：

import torch.nn as nn

model = nn.Sequential(
    nn.Embedding(num_embeddings=148, embedding_dim=768),
    nn.Flatten(),
    nn.Linear((768*148), 148),
    nn.ReLU(),
    nn.Linear(148, 3))

criterion = nn.CrossEntropyLoss()

optimizer = torch.optim.Adam(model.parameters())

通常像这样使用dataframe context column：

def tokenize_text(df, max_seq):
  return[
         tokenizer.encode(text, add_special_tokens=True)[:max_seq] for text in df.context.values
  ]

def pad_text(tokenized_text, max_seq):
  return np.array([el + [0] * (max_seq - len(el)) for el in tokenized_text])

在 cmets 之后：我认为 len(df.context) 是 148 和 updatenum_embeddings 为 148。我仍然不断收到 the same error。

非常感谢。

Answer 1

这是带有损失函数和优化器的模型定义：

import torch.nn as nn

model = nn.Sequential(
    nn.Embedding(num_embeddings=vocab_size, embedding_dim=50),
    nn.Flatten(),
    nn.Linear(50*len(X[0]), 100),
    nn.ReLU(),
    nn.Linear(100, 3))

criterion = nn.CrossEntropyLoss()

optimizer = torch.optim.Adam(model.parameters())

这是一个训练循环的粗略轮廓：

epochs = 10
for epoch in range(epochs):
    print(f'Epoch {epoch+1}/{epochs}')

    for x, y in data:
        y_hat = model(x)
        loss = criterion(y_hat, y)
        acc = (y_hat.argmax(1) == y).float().mean()

        print(f'loss: {loss}, accuracy: {acc}')

假设data 包含(x, y) 训练点，并且y 有一个包含地面实况类索引的单一维度。

---

其他 cmets：

• PyTorch 使用动态模型定义，它无法推断前一层的输出大小。在 Keras 中，您需要在第一层提供input_length。注意nn.Linear(50*len(X[0]), 100)（在Pytorch中）与Dense(100)（在Keras中）。

• 使用nn.CrossEntropyLoss时不需要nn.Softmax（它已经使用了nn.LogSoftmax）

• 供参考：nn.Sequential、nn.Linear、nn.Flatten 和 nn.Embedding。