Python 朴素贝叶斯分类器在电影评论语料库上训练以测试推文

Question

import nltk.classify.util
import csv
from nltk.classify import NaiveBayesClassifier
from nltk.corpus import movie_reviews

def word_feats(words):
    return dict([(word, True) for word in words])

negids = movie_reviews.fileids('neg')
posids = movie_reviews.fileids('pos')

negfeats = [(word_feats(movie_reviews.words(fileids=[f])), 'neg') for f in negids]
posfeats = [(word_feats(movie_reviews.words(fileids=[f])), 'pos') for f in posids]

negcutoff = len(negfeats)*3/4
poscutoff = len(posfeats)*3/4

trainfeats = negfeats[:negcutoff] + posfeats[:poscutoff]
testfeats = negfeats[negcutoff:] + posfeats[poscutoff:]
print 'train on %d instances, test on %d instances' % (len(trainfeats), len(testfeats))

classifier = NaiveBayesClassifier.train(trainfeats)
print 'accuracy:', nltk.classify.util.accuracy(classifier, testfeats)
classifier.show_most_informative_features()

我对 Python 非常陌生，正在尝试对推文进行情绪分析。我正在使用 NLTK 包中内置的朴素贝叶斯分类器。我正在电影评论语料库上对其进行测试，并希望在我使用 Tweepy 存储到 .txt 或 .csv 文件中的推文上进行测试。谁能帮助弄清楚如何从我的输出文件中的推文中测试这个分类器？谢谢！

Answer 1

只需加载推文：

f = open('tweets.txt', 'r')
data = f.readlines()
testfeats = word_feats([tweet.split(' ') for tweet in data]) # for file with tweets separated by line

然后你可以使用你的 word_feats 方法来提取特征（你可以使用 CountVectorizer 代替）。