从文本中提取国籍和国家

Question

我想使用 nltk 从文本中提取所有提到的国家和国籍，我使用 POS 标记来提取所有带有 GPE 标记的标记，但结果并不令人满意。

 abstract="Thyroid-associated orbitopathy (TO) is an autoimmune-mediated orbital inflammation that can lead to disfigurement and blindness. Multiple genetic loci have been associated with Graves' disease, but the genetic basis for TO is largely unknown. This study aimed to identify loci associated with TO in individuals with Graves' disease, using a genome-wide association scan (GWAS) for the first time to our knowledge in TO.Genome-wide association scan was performed on pooled DNA from an Australian Caucasian discovery cohort of 265 participants with Graves' disease and TO (cases) and 147 patients with Graves' disease without TO (controls). "

  sent = nltk.tokenize.wordpunct_tokenize(abstract)
  pos_tag = nltk.pos_tag(sent)
  nes = nltk.ne_chunk(pos_tag)
  places = []
  for ne in nes:
      if type(ne) is nltk.tree.Tree:
         if (ne.label() == 'GPE'):
            places.append(u' '.join([i[0] for i in ne.leaves()]))
      if len(places) == 0:
          places.append("N/A")

得到的结果是：

['Thyroid', 'Australian', 'Caucasian', 'Graves']

有些是国籍，有些只是名词。

那么我做错了什么还是有其他方法可以提取此类信息？

Answer 1

您可以将 Spacy 用于 NER。它提供了比 NLTK 更好的结果。

import spacy

nlp = spacy.load('en_core_web_sm')

doc = nlp(u"Apple is opening its first big office in San Francisco and California.")
print([(ent.text, ent.label_) for ent in doc.ents])

Answer 2

这是使用 NLTK 执行实体提取的 geograpy。它将所有地点和地点存储为地名词典。然后它在地名词典上执行查找以获取相关地点和位置。查看文档了解更多使用详情 -

from geograpy import extraction

e = extraction.Extractor(text="Thyroid-associated orbitopathy (TO) is an autoimmune-
mediated orbital inflammation that can lead to disfigurement and blindness. 
Multiple genetic loci have been associated with Graves' disease, but the genetic 
basis for TO is largely unknown. This study aimed to identify loci associated with 
TO in individuals with Graves' disease, using a genome-wide association scan 
(GWAS) for the first time to our knowledge in TO.Genome-wide association scan was 
performed on pooled DNA from an Australian Caucasian discovery cohort of 265 
participants with Graves' disease and TO (cases) and 147 patients with Graves' 
disease without TO (controls).")

e.find_entities()
print e.places()

Answer 3

因此，在卓有成效的 cmets 之后，我深入挖掘了不同的 NER 工具，以找到识别国籍和国家提及的最佳方法，并发现 SPACY 有一个 NORP 实体，可以有效地提取国籍。 https://spacy.io/docs/usage/entity-recognition

Answer 4

如果你想提取国家名称，你需要的是 NER 标注器，而不是 POS 标注器。

命名实体识别 (NER) 是信息提取的一个子任务，旨在定位文本中的元素并将其分类为预定义的类别，例如人名、组织、位置、时间表达、数量、货币价值、百分比等。

查看斯坦福 NER 标记器！

from nltk.tag.stanford import NERTagger
import os
st = NERTagger('../ner-model.ser.gz','../stanford-ner.jar')
tagging = st.tag(text.split())