【发布时间】:2010-09-12 06:05:20
【问题描述】:
def index_dir(self, base_path):
num_files_indexed = 0
allfiles = os.listdir(base_path)
#print allfiles
num_files_indexed = len(allfiles)
#print num_files_indexed
docnumber = 0
self._inverted_index = {} #dictionary
for file in allfiles:
self.documents = [base_path+file] #list of all text files
f = open(base_path+file, 'r')
lines = f.read()
# Tokenize the file into words
tokens = self.tokenize(lines)
docnumber = docnumber + 1
print 'docnumber', docnumber
for term in tokens:
# check if the key already exists in the dictionary, if yes,
# just add a new value for the key
#if self._inverted_index.has_key(term)
if term in sorted(self._inverted_index.keys()):
docnumlist = self._inverted_index.get(term)
docnumlist = docnumlist.append(docnumber)
else:
# if the key doesn't exist in dictionary, add the key (term)
# and associate the docnumber value with it.
self._inverted_index = self._inverted_index.update({term: docnumber})
#self._inverted_index[term] = docnumber
f.close()
print 'dictionary', self._inverted_index
print 'keys', self._inverted_index.keys()
return num_files_indexed
我正在从事一个信息检索项目,我们应该在其中爬取多个文本文件,对文件进行标记并将单词存储在倒排列表(字典)数据结构中。
例如:
doc1.txt:“狗跑了”
doc2.txt:“猫睡着了”
_inverted_index = {
'the': [0,1],
'狗':[0],
“跑”:[0],
'猫':[1],
“睡觉”:[1]
}
其中 0,1 是 docID
我收到以下错误: “非类型”对象没有属性键。第 95 行
非常感谢所有帮助。
【问题讨论】:
标签: python