我已经阅读了this 的 text1.similar("monstrous") 和 text1.concordance("monstrous")。
对于python中自然语言处理工具包的text1.concordance('monstrous')和text1.similar('monstrous')之间的区别，我无法得到满意的答案。
那么请您详细举例说明一下好吗？
【问题讨论】：
标签： python nltk text-processing
使用concordance(token) 可以为您提供围绕参数token 的上下文。它会显示token 出现的句子。
使用similar(token) 返回与token 出现在相同上下文中的单词列表。在这种情况下，上下文只是直接位于token 两侧的单词。
所以，查看 Moby Dick 文本 (text1)。我们可以检查'monstrous'的一致性
text1.concordance('monstrous')
# returns:
Displaying 11 of 11 matches:
ong the former , one was of a most monstrous size . ... This came towards us ,
ON OF THE PSALMS . " Touching that monstrous bulk of the whale or ork we have r
ll over with a heathenish array of monstrous clubs and spears . Some were thick
d as you gazed , and wondered what monstrous cannibal and savage could ever hav
that has survived the flood ; most monstrous and most mountainous ! That Himmal
they might scout at Moby Dick as a monstrous fable , or still worse and more de
th of Radney .'" CHAPTER 55 Of the Monstrous Pictures of Whales . I shall ere l
ing Scenes . In connexion with the monstrous pictures of whales , I am strongly
ere to enter upon those still more monstrous stories of them which are to be fo
ght have been rummaged out of this monstrous cabinet there is no telling . But
of Whale - Bones ; for Whales of a monstrous size are oftentimes cast up dead u
然后我们可以得到出现在与'monstrous' 相似的上下文中的单词列表。返回的第一行的上下文是'most _____ size'。
text1.similar('monstrous')
# returns:
determined maddens contemptible modifies abundant tyrannical puzzled
trustworthy impalpable gamesome curious mean pitiable untoward
christian subtly passing domineering uncommon true
如果我们使用单词 'true' 并检查它与 text.concordance('true') 的一致性，我们将返回单词“true”的 87 次使用中的前 25 次。这并不是非常有用，但 NLTK 确实提供了一个名为 common_contexts 的附加方法，该方法显示何时使用的单词列表共享相同的周围单词。
text1.common_contexts(['monstrous', 'true'])
# returns:
the_pictures
这个结果告诉我们，"the monstrous pictures" 和 "the true pictures" 都出现在 Moby Dick 中。
【讨论】：
我会举例说明：
text1.similar("monstrous")
将输出具有相似上下文的单词，例如word1 ______ word2。例如，它输出单词 doleful。如果你运行：
text1.concordance("monstrous")
你会在匹配的行中看到：
在洪水中幸存下来； 最可怕的和最多山的！ 那个希玛尔
如果你运行：
text1.concordance("doleful")
你会在匹配的行中看到：
ite 观点。有一个最悲惨最可笑的葬礼！ 大海
和
text1.common_contexts(["monstrous", "doleful"])
会输出monstrous和doleful的常用周边词“most”和“and”
最和
【讨论】：
Concordance(token) 为您提供使用令牌的上下文。
Similar(token) 为您提供出现在类似上下文中的其他词。
为了说明，这里有一个更一般的描述来近似它们的功能。
1)Concordance(token):
这会在令牌的左侧和右侧返回预定义数量的单词（让我们将此单词集合称为“Z”）。它会为您的令牌出现在文本中的每个实例执行此操作。
2)similar(token):
如果一个词出现在集合“Z”的词中的可能性很大，则将在此处列出该词。
【讨论】：
【讨论】：
我将在简·奥斯汀 1811 年的著作《理智与情感》（文本 2）中以“怪物”一词及其类似词的详尽示例进行解释：
common_contexts shared by "monstrous" and "very": ['a_pretty', 'am_glad', 'a_lucky', 'is_pretty', 'be_glad']
context: a_pretty
"a very pretty"
ct . She was short and plump , had a very pretty face , and the finest express
some delightful stew - ponds , and a very pretty canal ; and every thing , in
from Norland , Elinor had painted a very pretty pair of screens for her siste
"a monstrous pretty"
. " Now , Palmer , you shall see a monstrous pretty girl ." He immediately went
context: am_glad
"am very glad"
ouble you with my affairs . But I am very glad to find it was only my own fanc
be ordained in readiness ; and I am very glad to find things are so forward b
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
context: a_lucky
"a very lucky"
Colonel Brandon may be , Edward is a very lucky man .-- You will not mention t
"a monstrous lucky"
ou may tell your sister . She is a monstrous lucky girl to get him , upon my ho
context: is_pretty
"is very pretty"
t is only the Palmers . Charlotte is very pretty , I can tell you . You may se
" All this ," thought Elinor , " is very pretty ; but it can impose upon neit
eau of mine ." " Aye , aye , that is very pretty talking -- but it won ' t do
"is monstrous pretty"
k how you will like them . Lucy is monstrous pretty , and so good humoured and
context: be_glad
"be very glad"
trusting YOU ; indeed , I should be very glad of your advice how to manage in
o ever he might have , she should be very glad to have it all , you know , or
ant company , I am sure we should be very glad to come and stay with her for a
y ' s sister , my dear . I should be very glad to get her so good a mistress .
patience !-- And though one would be very glad to do a kindness by poor Mr . F
d , assuring her that Fanny would be very glad to see her , invited her to com
"be monstrous glad"
Jennings , " I am sure I shall be monstrous glad of Miss Marianne ' s company
common_contexts shared by "monstrous" and "so": ['am_glad', 'is_pretty']
context: am_glad
"am so glad"
delight in seeing them again . " I am so glad to see you !" said she , seating
eems very agreeable ." " Well -- I am so glad you do . I thought you would , h
small , was pleasing to her . " I am so glad we are got acquainted at last ,"
an be like it , to be sure ! And I am so glad your sister is going to be well
d otherwise have been learnt . " I am so glad to meet you ;" said Miss Steele
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
context: is_pretty
"is so pretty"
n it must be some other place that is so pretty I suppose ." When they were se
"is monstrous pretty"
k how you will like them . Lucy is monstrous pretty , and so good humoured and
common_contexts shared by "monstrous" and "exceedingly": ['be_glad', 'is_fond']
context: be_glad
"be exceedingly glad"
but , my dear Elinor , I shall be exceedingly glad to know more of it ; and I t
with her ; otherwise I should be exceedingly glad to do it . You know I am alw
"be monstrous glad"
Jennings , " I am sure I shall be monstrous glad of Miss Marianne ' s company
context: is_fond
"is exceedingly fond"
han you did ; not but what she is exceedingly fond of YOU , but so it happened
"is monstrous fond"
so scornfully ! for they say he is monstrous fond of her , as well he may . I s
common_contexts shared by "monstrous" and "heartily": ['am_glad', 'be_glad']
context: am_glad
"am heartily glad"
ney enough to attempt it ." " I am heartily glad of it ," he cried . " May she
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
context: be_glad
"be heartily glad"
saying now and then , ' I shall be heartily glad to hear she is well married .
"be monstrous glad"
Jennings , " I am sure I shall be monstrous glad of Miss Marianne ' s company
common_contexts shared by "monstrous" and "a": ['was_happy']
context: was_happy
"was a happy"
ne ' s behaviour as they travelled was a happy specimen of what future complais
"was monstrous happy"
thing of the kind . So then he was monstrous happy , and talked on some time ab
common_contexts shared by "monstrous" and "as": ['is_pretty']
context: is_pretty
"is as pretty"
, my dear , for shewing it me . It is as pretty a letter as ever I saw , and d
wing it me . It is as pretty a letter as ever I saw , and does Lucy ' s head a
"is monstrous pretty"
k how you will like them . Lucy is monstrous pretty , and so good humoured and
common_contexts shared by "monstrous" and "good": ['a_deal']
context: a_deal
"a good deal"
l ; but as she had already imbibed a good deal of Marianne ' s romance , witho
He and I have been at times thrown a good deal together , while you have been
th Mrs . Dashwood , so we shall be a good deal in Harley Street , I dare say ,
alone ," he replied , " for I have a good deal to say to you . This living of
"a monstrous deal"
e very genteel people . He makes a monstrous deal of money , and they keep thei
common_contexts shared by "monstrous" and "great": ['a_deal']
context: a_deal
"a great deal"
wn way , many cunning tricks , and a great deal of noise , as to outweigh all t
what you are doing . I have known a great deal of the trouble of annuities ; f
me as what belongs to this house . A great deal too handsome , in my opinion ,
inciple by my mother . I have seen a great deal of him , have studied his senti
, fat , elderly woman , who talked a great deal , seemed very happy , and rathe
een thirty and forty . He has seen a great deal of the world ; has been abroad
in all she does -- sometimes talks a great deal and always with animation -- bu
ey recommended the army . That was a great deal too smart for me . The law was
r , I dare say we should have seen a great deal of him in Somersetshire , if it
well married ! I hope you will be a great deal at Combe Magna . It is a sweet
old them it is all very true , and a great deal more . You will be delighted wi
ment at all ; indeed , she does me a great deal more harm than good , for I am
ings received the information with a great deal of joy , and many assurances of
ut it did not signify , for it was a great deal too far off to visit ; she hate
e in time . A thousand a - year is a great deal for a mother to give away , to
ied ; " but however there is still a great deal to be done . There is not a sto
not be forgotten .-- She must have a great deal to leave ." " Nothing at all ,
ious that everybody else should do a great deal ; and an offer from Colonel Bra
hinks just the same . He and I had a great deal of talk about it ; and the best
; so she told him directly ( with a great deal about sweet and love , you know
nd of whom therefore she must have a great deal to say , her mind was so much m
nfidant of himself , talked to her a great deal of the parsonage at Delaford ,
in the course of a twelve - month a great deal of instruction which I now feel
you , for as Colonel Brandon seems a great deal at home , nobody can tell what
"a monstrous deal"
e very genteel people . He makes a monstrous deal of money , and they keep thei
common_contexts shared by "monstrous" and "extremely": ['am_glad']
context: am_glad
"am extremely glad"
more than I can express ." " I am extremely glad to hear it , upon my word ; e
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
common_contexts shared by "monstrous" and "remarkably": ['a_pretty']
context: a_pretty
"a remarkably pretty"
lexion , regular features , and a remarkably pretty figure . Marianne was stil
"a monstrous pretty"
. " Now , Palmer , you shall see a monstrous pretty girl ." He immediately went
common_contexts shared by "monstrous" and "sweet": ['a_pretty']
context: a_pretty
"a sweet pretty"
at his house ; but they say it is a sweet pretty place ." " As vile a spot as
"a monstrous pretty"
. " Now , Palmer , you shall see a monstrous pretty girl ." He immediately went
common_contexts shared by "monstrous" and "vast": ['a_deal']
context: a_deal
"a vast deal"
s convenience ; and it HAS cost me a vast deal of money ." " More than you thi
othing but only civility ?-- I saw a vast deal more . Such kindness as fell to
here come the Richardsons . I had a vast deal more to say to you , but I must
ell , I am convinced that there is a vast deal of inconsistency in almost ever
"a monstrous deal"
e very genteel people . He makes a monstrous deal of money , and they keep thei
common_contexts shared by "monstrous" and "amazingly": ['am_glad']
context: am_glad
"am amazingly glad"
l be in no hurry to be gone . I am amazingly glad you did not keep to YOUR WORD
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
该输出由以下代码生成。
from nltk.book import *
from contextlib import redirect_stdout
import io
def get_similar(text, word):
f = io.StringIO()
with redirect_stdout(f):
text.similar(word)
return f.getvalue().split()
def get_concordance(text, word):
f = io.StringIO()
with redirect_stdout(f):
text.concordance(word, lines=1000)
return f.getvalue().split(sep='\n')[1:-1]
def get_common_contexts(text, word_list):
f = io.StringIO()
with redirect_stdout(f):
text.common_contexts(word_list)
return f.getvalue().split()
def print_full_similars(text, word):
word_concordances = get_concordance(text, word)
similars = get_similar(text, word)
for similar in similars:
similar_concordances = get_concordance(text, similar)
common_contexts = get_common_contexts(text, [word, similar])
print('common_contexts shared by "' + word + '" and "' + similar + '":', common_contexts)
for context in common_contexts:
print('\tcontext:', context)
before, after = context.split('_')
print('\t\t"' + before, similar, after + '"')
for concordance in similar_concordances:
if (before + ' ' + similar + ' ' + after).lower() in concordance.lower():
print('\t\t\t'+ concordance)
print('\t\t"' + before, word, after + '"')
for concordance in word_concordances:
if (before + ' ' + word + ' ' + after).lower() in concordance.lower():
print('\t\t\t'+ concordance)
print()
print_full_similars(text2, 'monstrous')
只需更改最后一行以获取其他示例，例如 Herman Melville 1851 的 Moby Dick：
print_full_similars(text1, 'monstrous')
【讨论】：