如何使用 sklearn.metrics.make_scorer 创建调整后的 R 平方记分器？答案

【问题标题】：How to create an Adjusted R-squared scorer using sklearn.metrics.make_scorer?如何使用 sklearn.metrics.make_scorer 创建调整后的 R 平方记分器？
【发布时间】：2021-12-22 08:48:50
【问题描述】：

我必须使用sklearn.metrics 的make_scorer 函数制作一个调整后的 R 平方可调用函数。调整后的 R 平方有一个参数，即特征的数量，我发现它很难编码。我做这个得分手的原因是我想在mlxtend.feature_selection.SequentialFeatureSelector中使用它。

import numpy as np
from sklearn.metrics import make_scorer
# x = y_true
# y = y_pred
# major_X = feature dataset

def Adj_r2(x, y, major_X):
zx = (x-np.mean(x))/np.std(x, ddof=1)
zy = (y-np.mean(y))/np.std(y, ddof=1)
r = np.sum(zx*zy)/(len(x)-1)
r2 = pow(r, 2)

# major_X.shape[1] gives to number of features that were used to make the prediction
return 1 - ((1-r2) * (len(x)-1)/(len(x)-major_X.shape[1]-1))


# Using make_scorer, to put it later in Sequential Feature Selector
scorer_adj_r2 = make_scorer(Adj_r2, greater_is_better=True)

【问题讨论】：

标签： python machine-learning scikit-learn scoring coefficient-of-determination

【解决方案1】：

您可以在this answer中建议使用签名func(estimator, X, y) 987654323。在您的情况下，自定义得分手定义将是：

import numpy as np

def r2_score_adj(estimator, X, y):

    y_pred = estimator.predict(X)

    if estimator.fit_intercept:
        rsquared = 1 - np.nansum((y - y_pred) ** 2) / np.nansum((y - np.nanmean(y)) ** 2)
        rsquared_adj = 1 - (X.shape[0] - 1) / (X.shape[0] - X.shape[1] - 1) * (1 - rsquared)

    else:
        rsquared = 1 - np.nansum((y - y_pred) ** 2) / np.nansum(y ** 2)
        rsquared_adj = 1 - X.shape[0] / (X.shape[0] - X.shape[1]) * (1 - rsquared)

    return rsquared_adj

相当于statsmodel's adjusted R-squared定义：

import statsmodels.api as sm
from sklearn.datasets import make_regression
from sklearn.linear_model import LinearRegression

X, y = make_regression(n_samples=100, n_features=10, noise=50, random_state=42)

# statsmodels with intercept
reg1 = sm.OLS(exog=sm.add_constant(X), endog=y).fit()
print(reg1.rsquared_adj)
# 0.9313017447410593

# scikit-learn with intercept
reg2 = LinearRegression(fit_intercept=True).fit(X, y)
print(r2_score_adj(reg2, X, y))
# 0.9313017447410593

# statsmodels without intercept
reg3 = sm.OLS(exog=X, endog=y).fit()
print(reg3.rsquared_adj)
# 0.9307276380801821

# scikit-learn without intercept
reg4 = LinearRegression(fit_intercept=False).fit(X, y)
print(r2_score_adj(reg4, X, y))
# 0.930727638080182

然后，您可以使用mlxtend's SequentialFeatureSelector的自定义得分手如下：

from mlxtend.feature_selection import SequentialFeatureSelector

sfs = SequentialFeatureSelector(
   estimator=LinearRegression(),
   k_features=2,
   forward=True,
   scoring=r2_score_adj,
   cv=2
)

sfs.fit(X, y)

print(sfs.subsets_)
# {1: {'feature_idx': (4,), 'cv_scores': array([0.11337299, 0.11996526]), 'avg_score': 0.1166691229065483, 'feature_names': ('4',)}, 2: {'feature_idx': (4, 9), 'cv_scores': array([0.20589701, 0.38117558]), 'avg_score': 0.2935362938943045, 'feature_names': ('4', '9')}}

【讨论】：