NLTK 分类的 ValueError

Question

我收到一条 ValueError 消息，但我不确定我做错了什么，或者我的 Python 安装是否有错误。我正在尝试开发一个测试来确定文档是虚构的还是非虚构的。我的代码是：

import nltk, re, string
from nltk.corpus import CategorizedPlaintextCorpusReader
corpus_root = './nltk_data/corpora/fiction'

fiction = CategorizedPlaintextCorpusReader(corpus_root, r'(\w+)/*.txt', cat_file='cat.txt')
fiction.categories()
['fic', 'nonfic']

documents = [(list(fiction.words(fileid)), category)
    for category in fiction.categories()
    for fileid in fiction.fileids(category)]

all_words=nltk.FreqDist(
    w.lower() 
    for w in fiction.words() 
    if w.lower() not in nltk.corpus.stopwords.words('english') and w.lower() not in string.punctuation)
word_features = all_words.keys()[:100]

def document_features(document): # [_document-classify-extractor]
    document_words = set(document) # [_document-classify-set]
    features = {}
    for word in word_features:
        features['contains(%s)' % word] = (word in document_words)
    return features
#print document_features(fiction.words('fic/11.txt'))

featuresets = [(document_features(d), c) for (d,c) in documents]
train_set, test_set = featuresets[100:], featuresets[:100]
classifier = nltk.NaiveBayesClassifier.train(train_set)

print nltk.classify.accuracy(classifier, test_set)
classifier.show_most_informative_features(5)

我得到以下回报： {'contains(girls)': True, 'contains(farm)': True, 'contains(new)': True, 'contains(left)': True, 'contains(days)': True, 'contains(work)': True, 'contains(stood)': True, 'contains("")': True, 'contains(subject)': True, 'contains(might)': True, 'contains(mrs)': False, 'contains(like)': True, 'contains(father)': True, 'contains(said)': True, 'contains(taken)': True, 'contains(little)': True, 'contains(every)': True, 'contains(first)': True, 'contains(."")': True, 'contains(uncle)': False, 'contains(close)': True, 'contains(week)': True, 'contains(women)': True, 'contains(interest)': True, 'contains(sally)': False, 'contains(body)': True, 'contains(life)': True, 'contains(home)': True, 'contains(nonfiction)': True, 'contains(spite)': True, 'contains(read)': True, 'contains(done)': True, 'contains(travis)': False, 'contains(place)': True, 'contains(woman)': True, 'contains(!"")': True, 'contains(old)': True, 'contains(boy)': True, 'contains(know)': True, 'contains(made)': True, 'contains(together)': True, 'contains(farmer)': True, 'contains(make)': True, 'contains(great)': True, 'contains(upon)': True, 'contains(men)': True, 'contains(hand)': True, 'contains(time)': True, 'contains(always)': True, 'contains(fiction)': True, 'contains(back)': True, 'contains(two)': True, 'contains(mother)': True, 'contains(would)': True, 'contains(country)': True, 'contains(put)': True, 'contains(,"")': True, 'contains(never)': True, 'contains(.")': True, 'contains(well)': True, 'contains(think)': True, 'contains(living)': True, 'contains(man)': True, 'contains(came)': True, 'contains(fruit)': True, 'contains(year)': True, 'contains(state)': True, 'contains(years)': True, 'contains(may)': True, 'contains(something)': True, 'contains(\x97)': True, 'contains(esther)': False, 'contains(,")': True, 'contains(get)': True, 'contains(children)': True, 'contains(many)': True, 'contains(better)': True, 'contains(away)': True, 'contains(spring)': True, 'contains(last)': True, 'contains(long)': True, 'contains(food)': True, 'contains(summer)': True, 'contains(girl)': True, 'contains(paper)': True, 'contains(city)': True, 'contains(could)': True, 'contains(come)': True, 'contains(part)': True, 'contains(see)': True, 'contains(wife)': True, 'contains(keep)': True, 'contains(along)': True, 'contains(even)': True, 'contains(people)': True, 'contains(best)': True, 'contains(good)': True, 'contains(day)': True, 'contains(season)': True, 'contains(one)': True} Traceback (most recent call last): File "fiction.py", line 44, in <module> classifier = nltk.NaiveBayesClassifier.train(train_set) File "/usr/local/lib/python2.6/dist-packages/nltk/classify/naivebayes.py", line 214, in train label_probdist = estimator(label_freqdist) File "/usr/local/lib/python2.6/dist-packages/nltk/probability.py", line 898, in __init__ LidstoneProbDist.__init__(self, freqdist, 0.5, bins) File "/usr/local/lib/python2.6/dist-packages/nltk/probability.py", line 782, in __init__ 'must have at least one bin.') ValueError: A ELE probability distribution must have at least one bin.

我应该收到每个类别中最常用词的概率，但我得到了错误。

Answer 1

你已经很久没有问过了；但是，我敢打赌您的“功能集”少于 100 条记录。你会得到这个，因为你的 test_set 可能是一个空列表。