过采样时如何保持/扩展索引答案

【问题标题】：How to keep/extend index when oversample过采样时如何保持/扩展索引
【发布时间】：2021-03-09 07:57:34
【问题描述】：

我有一个这样的数据框，我想对列“角色”进行过采样（在实际情况下，行/列的数量比这个最小示例大得多）

                 role  value
pop_13vdpn1_site_1  1   1
pop_13vdpn1_site_1  1   1
pop_13vdpn1_site_1  1   2
pop_13vdpn1_site_1  1   1
pop_13vdpn1_site_1  1   1
pop_13vdpn1_site_1  1   2
pop_13vdpn1_site_1  1   1
pop_13vdpn1_site_1  2   1
pop_13vdpn1_site_1  2   1
pop_13vdpn1_site_1  2   1
pop_13vdpn1_site_2  2   1
pop_13vdpn1_site_2  2   2
pop_13vdpn1_site_2  2   1
pop_13vdpn1_site_2  2   1
pop_13vdpn1_site_2  2   1
pop_13vdpn1_site_2  2   1
pop_13vdpn1_site_2  2   1
pop_13vdpn1_site_2  2   1
pop_13vdpn1_site_2  2   1
pop_13vdpn1_site_3  2   1
[...........]

Index: 20 entries, pop_13vdpn1_site_1 to pop_13vdpn1_site_1
Data columns (total 2 columns):
role     20 non-null int64
value    20 non-null int64

这就是我正在做的：

X,y = smote.fit_sample(df,df[['role']])
X
       role value
0   1   1
1   1   1
2   1   2
3   1   1
4   1   1
5   1   2
6   1   1
7   2   1
8   2   1
[.........]

它可以工作，但问题是我需要保留索引（pop_13vdpn1_site_1 等）这可能吗？

【问题讨论】：

标签： python pandas imbalanced-data oversampling smote

【解决方案1】：

我终于找到了一种解决方法（可能不是最佳的）

from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
df_tmp = df.reset_index()
df_tmp['index'] = le.fit_transform(df_tmp['index'])
aa,bb = smote.fit_sample(df_tmp,df_tmp[['role']])
aa['index'] = le.inverse_transform(aa['index'])
aa.set_index('index')

【讨论】：

有趣！我遇到了非常相似的情况，你的建议就是我现在要尝试的。感谢分享！

【解决方案2】：

首先，您需要处理 df 并将您的特征和目标标签拆分为 X_train 和 y_train。

现在您可以进行过采样了：

X_train_over, y_train_over = smote.fit_sample(X_train, y_train)

最后从上面的输出创建一个数据框。例如，

X = pd.DataFrame(X_train_over, columns=X_train.columns)
y = pd.DataFrame(y_train_over, columns=y_train.columns)

【讨论】：

嗨，Giorgos，但是，如果我这样做，我会得到 X 和 y 的 NaN 值
@psagrera 如果不提供index 参数，输出是什么？
角色值 0 1 1

【解决方案3】：

以下应该这样做。

import io
import pandas as pd
import numpy as np
from imblearn.over_sampling import SMOTE

示例数据。

df = pd.read_csv(io.StringIO("""
role  value
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 1 2
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 1 2
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 2 1
    pop_13vdpn1_site_1 2 1
    pop_13vdpn1_site_1 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 2
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_3 2 1
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 1 2
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 1 2
    pop_13vdpn1_site_1 1 1
    pop_13vdpn1_site_1 2 1
    pop_13vdpn1_site_1 2 1
    pop_13vdpn1_site_1 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 2
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_2 2 1
    pop_13vdpn1_site_3 2 1
"""), sep="\s+", engine="python")

df = df.reset_index()

形状应该是 (40, 3):

df.shape

Smote 接受数组，所以我们需要定义 x 和 y 值。

X_train = np.array(df['role']).reshape(40,1)
y_train = np.array(df['value']).reshape(40,)

Smote 在行动：

from imblearn.over_sampling import SMOTE
sm = SMOTE(random_state=42)
X,y = sm.fit_resample(X_train,y_train)

将给定的X 和y 放入DataFrame：

ndf = pd.DataFrame({'role':X.reshape(68,), 'value':y})

重新命名原来的名字。

ndf['name'] = ndf['role'].apply(lambda x: 'pop_13vdpn1_site_'+str(x))

看看数据是否更平衡。

from collections import Counter
Counter(df['role'])
Counter(ndf['role'])

【讨论】：