MT5ForConditionalGeneration 与 Pytorch-lightning 给出 attribute_error答案

【问题标题】：MT5ForConditionalGeneration with Pytorch-lightning gives attribute_errorMT5ForConditionalGeneration 与 Pytorch-lightning 给出 attribute_error
【发布时间】：2022-03-12 10:54:14
【问题描述】：

几天都无法处理这个问题，因为我是 NLP 新手，实际的解决方案可能非常简单

class QAModel(pl.LightningDataModule):

  def __init__(self):
    super().__init__()
    self.model = MT5ForConditionalGeneration.from_pretrained(MODEL_NAME, return_dict=True)

  def forward(self, input_ids, attention_mask, labels=None):
    output = model(
        input_ids=input_ids,
        attention_mask=attention_mask,
        labels=labels
    )

    return output.loss, output.logits
  
  def training_step(self, batch, batch_idx):
    input_ids = batch['input_ids']
    attention_mask = batch['attention_mask']
    labels = batch['labels']
    loss, outputs = self(input_ids, attention_mask, labels)
    self.log('train_loss', loss, prog_bar=True, logger=True)
    return loss
  
  def validation_step(self, batch, batch_idx):
    input_ids = batch['input_ids']
    attention_mask = batch['attention_mask']
    labels = batch['labels']
    loss, outputs = self(input_ids, attention_mask, labels)
    self.log('val_loss', loss, prog_bar=True, logger=True)
    return loss

  def test_step(self, batch, batch_idx):
    input_ids = batch['input_ids']
    attention_mask = batch['attention_mask']
    labels = batch['labels']
    loss, outputs = self(input_ids, attention_mask, labels)
    self.log('test_loss', loss, prog_bar=True, logger=True)
    return loss

  def configure_optimizers(self):
    return AdamW(self.parameters(), lr=0.0001)
model = QAModel()
from pytorch_lightning.callbacks import ModelCheckpoint

checkpoint_callback = ModelCheckpoint(
    dirpath='/content/checkpoints',
    filename='best-checkpoint',
    save_top_k=1,
    verbose=True,
    monitor='val_loss',
    mode='min'
)
trainer = pl.Trainer(
    checkpoint_callback=checkpoint_callback,
    max_epochs=N_EPOCHS,
    gpus=1,
    progress_bar_refresh_rate=30
)
trainer.fit(model, data_module)

运行这段代码给了我 AttributeError：“QAModel”对象没有属性“automatic_optimization” 在 fit() 函数之后可能问题出在 MT5ForConditionalGeneration 中，因为在将其传递给 function() 后，我们遇到了同样的错误

【问题讨论】：

标签： optimization pytorch pytorch-lightning

【解决方案1】：

尝试继承pl.LightingModule 而不是pl.LightningDataModule。这是定义模型类的正确选择。

【讨论】：