适用于浮点数但不适用于字符串的条件熵函数

Question

我正在 Python 中实现一个 ID3 决策树，但我遇到了条件熵问题。当输入数组是字符串时，我的结果不是我所期望的。

https://en.wikipedia.org/wiki/Conditional_entropy

Compute the conditional entropy of y given x. The conditional entropy H(Y|X) means average entropy of children nodes, given attribute X. Refer to https://en.wikipedia.org/wiki/Information_gain_in_decision_trees
Input:
  X: a list of values , a numpy array of int/float/string values. The size of the array means the number of instances/examples. X contains each instance's attribute value. 
  Y: a list of values, a numpy array of int/float/string values. Y contains each instance's corresponding target label. For example X[0]'s target label is Y[0]
Output:
  ce: the conditional entropy of y given x, a float scalar

我在这里阅读了许多相关问题，我想我理解我想要做什么，但我是 Python 新手，我一定有什么问题。

import math
import numpy as np
from collections import Counter
class Tree(object):
     def entropy(Y):
        e = 0.
        total = len(Y)
        if total <= 1:
            return 0
        for num in Counter(Y).values():
            p = num/total
            e -= p * math.log2(p)
        return e 

     def conditional_entropy(Y,X):
        def indices(v,Y):
              return [i for i, j in enumerate(Y) if j == v]
        ce = 0.
        print("\nY = {}".format(Y))
        print("X = {}".format(X))
        for label in Counter(Y).keys():
            print("Checking label {}".format(label))
            sv = [X[i] for i in indices(label,Y)]
            e = Tree.entropy(sv)
            print("Subset entropy = {}".format(e))
            ce += e * len(sv)/total
            print("Cond.entropy so far = {}".format(ce))
         print("Finished. Result: {}".format(ce))
         return ce

还有一些测试：

def test_conditional_entropy():
    '''(6 points) conditional entropy '''

    y = np.array([0.,0.])
    x = np.array([1.,1.])
    ce = Tree.conditional_entropy(y,x)
    assert np.allclose(ce, 0., atol = 1e-3) 

    y = np.array([0.,1.])
    x = np.array([1.,2.])
    ce = Tree.conditional_entropy(y,x)
    assert np.allclose(ce, 0., atol = 1e-3)

    y = np.array([0.,1.,0.,1.])
    x = np.array([1.,4.,1.,4.])
    ce = Tree.conditional_entropy(y,x)
    assert np.allclose(ce, 0., atol = 1e-3) 

    y = np.array([0.,1.,0.,1.])
    x = np.array([1.,1.,4.,4.])
    ce = Tree.conditional_entropy(y,x)
    assert np.allclose(ce, 1., atol = 1e-3) 

    y = np.array(['apple','orange'])
    x = np.array(['good','good'])
    ce = Tree.conditional_entropy(y,x)
    assert np.allclose(ce, 1., atol = 1e-3) 

    y = np.array(['apple','orange'])
    x = np.array(['good','bad'])
    ce = Tree.conditional_entropy(y,x)
    assert np.allclose(ce, 0., atol = 1e-3) 

    y = np.array(['apple','orange','pineapple','banana'])
    x = np.array(['a','a','a','a'])
    ce = Tree.conditional_entropy(y,x)
    assert np.allclose(ce, 2., atol = 1e-3) 

    y = np.array(['apple','orange','pineapple','banana'])
    x = np.array(['a','a','b','b'])
    ce = Tree.conditional_entropy(y,x)
    assert np.allclose(ce, 1., atol = 1e-3)

我的结果：

FAIL: (6 points) conditional entropy
----------------------------------------------------------------------
Traceback (most recent call last):
  File "case.py", line 197, in runTest
    self.test(*self.arg)
  File "test2.py", line 36, in test_conditional_entropy
    assert np.allclose(ce, 1., atol = 1e-3)
AssertionError:

第 36 行是断言测试第一次失败的地方。这是第一个使用字符串的测试。为简洁起见，我删除了一些成功的结果；使用浮点数组的所有四个测试都得出了正确的结果。我的 entropy() 方法的所有单元测试也都通过了。我的输入是否混淆了？我觉得我缺少一些基本的东西。

-------------------- >> begin captured stdout << ---------------------

Y = [0. 1. 0. 1.]
X = [1. 1. 4. 4.]
Checking label 0.0
Subset entropy = 1.0
Cond.entropy so far = 0.5
Checking label 1.0
Subset entropy = 1.0
Cond.entropy so far = 1.0
Finished. Result: 1.0

Y = ['apple' 'orange']
X = ['good' 'good']
Checking label apple
Subset entropy = 0
Cond.entropy so far = 0.0
Checking label orange
Subset entropy = 0
Cond.entropy so far = 0.0
Finished. Result: 0.0

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 1 test in 0.003s

FAILED (failures=1)

Answer 1

嗯，我是对的，我的输入混淆了。我已经切换了 X 和 Y。现在可以使用（删除打印语句）：

   def conditional_entropy(Y,X):
        def indices(v,X):
              return [i for i, j in enumerate(X) if j == v]
        ce = 0.
        total = len(Y)
        for label in Counter(X).keys():
            sv = [Y[i] for i in indices(label,X)]
            e = Tree.entropy(sv)
            ce += e * len(sv)/total
        return ce