如何在 Pearson 相关计算期间摆脱“ValueError：连接轴的所有输入数组维度必须完全匹配”？答案

【问题标题】：How to get rid of 'ValueError: all the input array dimensions for the concatenation axis must match exactly' during Pearson Correlation calculation?如何在 Pearson 相关计算期间摆脱“ValueError：连接轴的所有输入数组维度必须完全匹配”？
【发布时间】：2020-08-02 07:45:58
【问题描述】：

我正在尝试根据here 提供的要点计算Pearson Correlation。奇怪的是收到ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 52 and the array at index 1 has size 1 错误（数据框有 52 条记录）。

这里是提供的功能：

def cor_selector(X, y, num_feats):
    cor_list = []
    feature_name = X.columns.tolist()
    # calculate the correlation with y for each feature
    for i in X.columns.tolist():
        cor = np.corrcoef(X[i], y)[0, 1] # error happens during the 2nd call to here
        cor_list.append(cor)
    # replace NaN with 0
    cor_list = [0 if np.isnan(i) else i for i in cor_list]
    # feature name
    cor_feature = X.iloc[:, np.argsort(np.abs(cor_list))[-num_feats:]].columns.tolist()
    # feature selection? 0 for not select, 1 for select
    cor_support = [True if i in cor_feature else False for i in feature_name]
    return cor_support, cor_feature

这是我的脚本：

df = pd.read_csv(DATA_CSV) # shape: (52, 5)
X = df[['a', 'b', 'c']]
y = df[['d']]
num_feats = 3
cor_support, cor_feature = cor_selector(X, y, num_feats)
print(str(len(cor_feature)), 'selected features')

完整的堆栈跟踪：

Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevd.py", line 1438, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/talha/PycharmProjects/covid19/store_data.py", line 275, in <module>
    cor_support, cor_feature = cor_selector(X, y, num_feats)
  File "/Users/talha/PycharmProjects/covid19/store_data.py", line 254, in cor_selector
    cor = np.corrcoef(X[i], y)[0, 1]
  File "<__array_function__ internals>", line 6, in corrcoef
  File "/Users/talha/.local/share/virtualenvs/covid19-g87yyZJK/lib/python3.7/site-packages/numpy/lib/function_base.py", line 2526, in corrcoef
    c = cov(x, y, rowvar)
  File "<__array_function__ internals>", line 6, in cov
  File "/Users/talha/.local/share/virtualenvs/covid19-g87yyZJK/lib/python3.7/site-packages/numpy/lib/function_base.py", line 2390, in cov
    X = np.concatenate((X, y), axis=0)
  File "<__array_function__ internals>", line 6, in concatenate

【问题讨论】：

第一步是确定哪条线路产生了问题，以及该调用的输入。你知道我们所说的full traceback 是什么意思吗？
在第二次执行 cor = np.corrcoef(X[i], y)[0, 1] 期间发生。完整的堆栈跟踪已添加到 OP。
试试np.concatenate((X[i], y), axis=1)，检查X[i]和y的形状。

标签： pandas numpy scipy feature-selection pearson-correlation

【解决方案1】：

您似乎将索引 0 处的系列和索引 1 处的数据框传递给 np.corrcoef。在您的脚本中，将y = df[['d']] 更改为y = df['d']，它应该可以工作。

【讨论】：