根据相关值选择特定特征

Question

我正在使用来自 Kaggle 的 Housing train.csv 数据进行预测。
https://www.kaggle.com/c/house-prices-advanced-regression-techniques/data?select=train.csv

我正在尝试生成相关性，并且仅将与 SalePrice 相关的特征保持在 0.5 到 0.9 之间。我试图使用这个函数来过滤其中的一些，但我只删除了高于 0.9 的相关值。 如何更新此函数以仅保留生成相关热图所需的那些特定特征？

data = train
corr = data.corr()
columns = np.full((corr.shape[0],), True, dtype=bool)
for i in range(corr.shape[0]):
    for j in range(i+1, corr.shape[0]):
        if corr.iloc[i,j] >= 0.9:
            if columns[j]:
                columns[j] = False
selected_columns = data.columns[columns]
data = data[selected_columns]

Answer 1

import pandas as pd

data = pd.read_csv('train.csv')
col = data.columns

c  = [i for i in col if data[i].dtypes=='int64' or data[i].dtypes=='float64']   # dropping columns as dtype == object
main_col = ['SalePrice']        # column with which we have to compare correlation

corr_saleprice = data.corr().filter(main_col).drop(main_col)    

c1 =(corr_saleprice['SalePrice']>=0.5) & (corr_saleprice['SalePrice']<=0.9)
c2 =(corr_saleprice['SalePrice']>=-0.9) & (corr_saleprice['SalePrice']<=-0.5)

req_index= list(corr_saleprice[c1 | c2].index)   # selecting column with given criteria

#req_index.append('SalePrice')      #if you want SalePrice column in your final dataframe too , uncomment this line

data = data[req_index]  

data

同样使用 for 循环也不是那么有效，直接实现是有利的。我希望这就是你想要的！

要生成热图，您可以使用以下代码：

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

a  =data.corr()
mask = np.triu(np.ones_like(a, dtype=np.bool))
plt.figure(figsize=(10,10))
_ = sns.heatmap(a,cmap=sns.diverging_palette(250, 20, n=250),square=True,mask=mask,annot=True,center=0.5)