如何根据来自 Pytorch Lightning 中的 train_dataloader 方法的权重为我的损失函数应用不同的权重？答案

【问题标题】：How can I apply different weights for my loss funciton based on the ones coming from my train_dataloader method in Pytorch Lightning?如何根据来自 Pytorch Lightning 中的 train_dataloader 方法的权重为我的损失函数应用不同的权重？
【发布时间】：2022-07-22 18:39:02
【问题描述】：

所以基本上，我使用的是 Pytorch Lightning 模块中的类。我的问题是我正在使用 Pytorch Dataloader 加载数据：

def train_dataloader(self):
    train_dir = f"{self.img_dir_gender}/train"
    # train_transforms: from PIL to TENSOR + DATA AUG
    train_transforms = T.Compose([
        T.ToTensor(),
        # T.Pad(25, padding_mode='symmetric'),
        # T.RandomHorizontalFlip(),
        # T.RandomVerticalFlip()
    ])
    train_dataset = ImageFolder(train_dir, transform=train_transforms)

    print(train_dataset.class_to_idx)
    print(Counter(train_dataset.targets))

    # oversampling giving more weight to minority classes
    class_weights = Counter(train_dataset.targets)
    class_weights_adjusted = [0] * len(train_dataset)
    for idx, (data, label) in enumerate(train_dataset):
    # inverse gives more weight to minority classes
        class_weight = 1 / class_weights[label]
        class_weights_adjusted[idx] = class_weight
    sampler = WeightedRandomSampler(class_weights_adjusted, num_samples=self.num_samples , replacement=True)

    train_loader = DataLoader(train_dataset, batch_size=self.hparams.batch_size, num_workers=4, sampler=sampler, shuffle=False)
    return train_loader

在那里我设法检索了我的班级权重并执行了一些过采样：

但是，我无法设法检索这些权重，例如，将它们取反，然后将它们传递给我的 training_step 和 val_step 方法中的 cross_entropy 损失函数，目的是解决我的 val 中的类不平衡问题数据集：

def training_step(self, batch, batch_idx):
    # torch.Size([bs, 3, 224, 224])
    # x = batch["pixel_values"]
    # torch.Size([bs])
    # y = batch["labels"]
    x, y = batch
    # unfreeze after a certain number of epochs
    # self.trainer.current_epoch >=

    # meaning it will not keep a graph with grads for the backbone (memory efficient)
    if self.trainer.current_epoch < self.hparams.unfreeze_epoch:
        with torch.no_grad():
            features = self.backbone(x)
    else:
        features = self.backbone(x)
    preds = self.finetune_layer(features)
    # pred_probs = softmax(preds, dim=-1)
    # pred_labels = torch.argmax(pred_probs, dim=-1)
    train_loss = cross_entropy(preds, y, weight=?)
    self.log("train_loss", train_loss, on_step=True, on_epoch=True, prog_bar=True, logger=True)
    self.log("train_accuracy", self.train_accuracy(preds, y), on_step=True, on_epoch=True, prog_bar=True, logger=True)
    self.log("train_f1_score", self.train_f1(preds, y), on_step=True, on_epoch=True, prog_bar=True, logger=True)
    #self.log("train_accuracy", self.train_accuracy(preds, y), prog_bar=True)
    #self.log("train_precision", self.train_precision(preds, y), prog_bar=True)
    #self.log("train_recall", self.train_recall(preds, y), prog_bar=True)
    #self.log("train_f1", self.train_f1(preds, y), prog_bar=True)
    return train_loss

所以我知道我应该在cross_entropy 函数中使用weight= 参数，但是如何从我的训练数据集中检索我的类权重？

如果我应该添加一些说明，请告诉我。

【问题讨论】：

标签： python deep-learning cross-entropy imbalanced-data pytorch-lightning

【解决方案1】：

你可以：

dm = DataModule()
# write your weights getter function in your pl.LightningDataModule
weights = dm.get_weights()
# where your loss function is set under your pl.LightningModule's init 
#
#        self.loss = nn.CrossEntropyLoss(weights=weights)) 
#
# and then called under training_step as self.loss(preds, y)
model = model(weights) 
trainer.fit(model, dm)

无需一直将权重传递给您的损失函数

【讨论】：