【问题标题】:Trouble getting regular expression to return given scraped website's text无法让正则表达式返回给定的抓取网站的文本
【发布时间】:2017-05-09 17:55:10
【问题描述】:

所以我抓取了一个网站,现在我正在尝试使用 RE 提取文本的某些部分。

source = urllib.request.urlopen('http://www.nasdaq.com/article/2017-small-cap-biotech-watchlist-takeaways-for-investors-cm739688')
soup = bs.BeautifulSoup(source,'html.parser')

for paragraph in soup.find_all('p'):
    print(paragraph.text)

产生类似:

In a panel discussion moderated by Maxim Group's senior managing director and the head of healthcare research Jason Kolbert, Raghuram "Ram" Selvaraju of Rodman & Renshaw; George Zavoico, senior equity analyst at JonesTrading Institutional Services; and Mike King, managing director and senior biotechnology analyst at JMP Securities, detailed their investment theses for the companies that made the cut.
King chose nine companies for the list, including OncoMed Pharmaceuticals Inc. (OMED:NASDAQ) , a company "we've had a long-term passion for." He considers OncoMed a "leader in the cancer stem cell space," with a number of clinical assets including a handful that are proprietary, and many partnered with larger-cap pharmas and biotechs. Among the candidates slated for either clinical trials or data readouts in 2017 is demcizumab, in pancreatic cancer. "We think it will be a key catalyst for the shares," King said. Other assets include tarextumab, for a small cell lung cancer; DLL4/VEG4, also in non-small cell lung cancer; and "a new asset that was previously known as I/O#2. . .a very highly validated immuno-oncology target called TIGIT," in development in partnership with Celgene Corp. (CELG:NASDAQ).
King called Syndax Pharmaceuticals Inc. (SNDX:NASDAQ) "a hybrid story. " The company is a player in the immuno-oncology space as well as in the "broader" cancer world with a drug called entinostat, which is in Phase 3 in metastatic breast cancer. Readouts on overall survival aren't expected until 2019, but King is "optimistic about Syndax's chances for success in the Phase 3 trial." The company is also in collaborations with major pharmas in the "immune checkpoint space," with data expected to "roll out over the course of 2017."
Syros Pharmaceuticals (SYRS:NASDAQ) is an early-stage company, with a ~$250M market cap, that King considers a leader in gene regulation, with a concept called super-enhancers. The company both develops its "own internal candidates to some of the highly validated targets," as well as repurposes assets that may have been abandoned for indications like blood disorders. Syros might be "a quiet story the first half of 2017, but I think in the second half of 2017, we'll start to see some fireworks based on the data that will read out through key assets, SY-1365 and SY-1425."

我正在尝试提取如下所示的股票代码:

(OMED:NASDAQ)
(SYRS:NASDAQ)

我在努力

pattern = re.compile(r':\(\w+\)')
for paragraph in soup.find_all('p'):
    print(paragraph.find_all(text =pattern))

但它正在屈服

[]
[]
[]
[]
[]

编辑:

也试过了 pattern = re.compile(r'\(\w+:\w+\)' ) 和 `pattern = re.compile(r'([A-Z]+:[A-Z]+)')'

两者都产生了这样的结果:

[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
['OncoMed Pharmaceuticals Inc. (OMED:NASDAQ)', ' , a company "we\'ve had a long-term passion for." He considers OncoMed a "leader in the cancer stem cell space," with a number of clinical assets including a handful that are proprietary, and many partnered with larger-cap pharmas and biotechs. Among the candidates slated for either clinical trials or data readouts in 2017 is demcizumab, in pancreatic cancer. "We think it will be a key catalyst for the shares," King said. Other assets include tarextumab, for a small cell lung cancer; DLL4/VEG4, also in non-small cell lung cancer; and "a new asset that was previously known as I/O#2. . .a very highly validated immuno-oncology target called TIGIT," in development in partnership with Celgene Corp. (CELG:NASDAQ).']
['King called Syndax Pharmaceuticals Inc. (SNDX:NASDAQ) "a hybrid story. " The company is a player in the immuno-oncology space as well as in the "broader" cancer world with a drug called entinostat, which is in Phase 3 in metastatic breast cancer. Readouts on overall survival aren\'t expected until 2019, but King is "optimistic about Syndax\'s chances for success in the Phase 3 trial." The company is also in collaborations with major pharmas in the "immune checkpoint space," with data expected to "roll out over the course of 2017."']
['Syros Pharmaceuticals (SYRS:NASDAQ) is an early-stage company, with a ~$250M market cap, that King considers a leader in gene regulation, with a concept called super-enhancers. The company both develops its "own internal candidates to some of the highly validated targets," as well as repurposes assets that may have been abandoned for indications like blood disorders. Syros might be "a quiet story the first half of 2017, but I think in the second half of 2017, we\'ll start to see some fireworks based on the data that will read out through key assets, SY-1365 and SY-1425."']

【问题讨论】:

    标签: python regex bs4


    【解决方案1】:

    尝试:

    pattern = re.compile(r'\(\w+:\w+\)' )
    # This will match any words in range of [0-9a-zA-Z]
    

    示例在: https://regex101.com/r/P2yqzF/3

    也许re.findall(pattern, paragraph.text) 会解决你的第二个问题

    【讨论】:

    • 试过了,但奇怪的是我仍然收到很多文本。我用结果更新了 OP。
    • 我更新了链接,我只捕获了 regex101 上的正确值,你捕获了什么?
    • 我更新了 OP,结果我尝试了你的建议。我想知道这是否是Python的东西?我也将更新我抓取的网站。
    • 所有建议的答案都暗示了相同的正则表达式,所以也许你的错在别处?
    • 我用废弃的网站和用于废弃网站的代码更新了 OP。我还是想不通。
    【解决方案2】:

    试试

    pattern = re.compile(r'\([A-Z]+:[A-Z]+\)')
    

    寻找<open parantheses><caps letter><colon><caps letter><close parantheses>

    然后,为了找到字符串,你可以做re.findall(paragraph)

    (假设段落是一个字符串)

    【讨论】:

    • 奇怪的是,我仍然收到很多文本,类似于尝试Ludisposed 建议时的输出。我更新了 OP。
    • 使用regexr.com,我输入了你的段落和我的正则表达式,并且只捕获了目标值;段落对象的“类型”是什么?
    • 我已经用我用来抓取网站的代码和网站更新了 OP。也许我做错了什么?
    • for paragraph in soup.find_all('p'): print(type(paragraph)) 的结果是什么?
    【解决方案3】:
    paragraphs = '''\
    In a panel discussion moderated by Maxim Group's senior managing director and the head of healthcare research Jason Kolbert, Raghuram "Ram" Selvaraju of Rodman & Renshaw; George Zavoico, senior equity analyst at JonesTrading Institutional Services; and Mike King, managing director and senior biotechnology analyst at JMP Securities, detailed their investment theses for the companies that made the cut.
    King chose nine companies for the list, including OncoMed Pharmaceuticals Inc. (OMED:NASDAQ) , a company "we've had a long-term passion for." He considers OncoMed a "leader in the cancer stem cell space," with a number of clinical assets including a handful that are proprietary, and many partnered with larger-cap pharmas and biotechs. Among the candidates slated for either clinical trials or data readouts in 2017 is demcizumab, in pancreatic cancer. "We think it will be a key catalyst for the shares," King said. Other assets include tarextumab, for a small cell lung cancer; DLL4/VEG4, also in non-small cell lung cancer; and "a new asset that was previously known as I/O#2. . .a very highly validated immuno-oncology target called TIGIT," in development in partnership with Celgene Corp. (CELG:NASDAQ).
    King called Syndax Pharmaceuticals Inc. (SNDX:NASDAQ) "a hybrid story. " The company is a player in the immuno-oncology space as well as in the "broader" cancer world with a drug called entinostat, which is in Phase 3 in metastatic breast cancer. Readouts on overall survival aren't expected until 2019, but King is "optimistic about Syndax's chances for success in the Phase 3 trial." The company is also in collaborations with major pharmas in the "immune checkpoint space," with data expected to "roll out over the course of 2017."
    Syros Pharmaceuticals (SYRS:NASDAQ) is an early-stage company, with a ~$250M market cap, that King considers a leader in gene regulation, with a concept called super-enhancers. The company both develops its "own internal candidates to some of the highly validated targets," as well as repurposes assets that may have been abandoned for indications like blood disorders. Syros might be "a quiet story the first half of 2017, but I think in the second half of 2017, we'll start to see some fireworks based on the data that will read out through key assets, SY-1365 and SY-1425."'''
    
    import re
    exp = re.compile(r'(\([A-Z]+\:[A-Z]+\))')
    
    for para in paragraphs.split('\n'):
        print (para)
        print (exp.findall(para))
    

    结果:

    In a panel discussion moderated by Maxim Group's senior managing director and the head of healthcare research Jason Kolbert, Raghuram "Ram" Selvaraju of Rodman & Renshaw; George Zavoico, senior equity analyst at JonesTrading Institutional Services; and Mike King, managing director and senior biotechnology analyst at JMP Securities, detailed their investment theses for the companies that made the cut.
    []
    King chose nine companies for the list, including OncoMed Pharmaceuticals Inc. (OMED:NASDAQ) , a company "we've had a long-term passion for." He considers OncoMed a "leader in the cancer stem cell space," with a number of clinical assets including a handful that are proprietary, and many partnered with larger-cap pharmas and biotechs. Among the candidates slated for either clinical trials or data readouts in 2017 is demcizumab, in pancreatic cancer. "We think it will be a key catalyst for the shares," King said. Other assets include tarextumab, for a small cell lung cancer; DLL4/VEG4, also in non-small cell lung cancer; and "a new asset that was previously known as I/O#2. . .a very highly validated immuno-oncology target called TIGIT," in development in partnership with Celgene Corp. (CELG:NASDAQ).
    ['(OMED:NASDAQ)', '(CELG:NASDAQ)']
    King called Syndax Pharmaceuticals Inc. (SNDX:NASDAQ) "a hybrid story. " The company is a player in the immuno-oncology space as well as in the "broader" cancer world with a drug called entinostat, which is in Phase 3 in metastatic breast cancer. Readouts on overall survival aren't expected until 2019, but King is "optimistic about Syndax's chances for success in the Phase 3 trial." The company is also in collaborations with major pharmas in the "immune checkpoint space," with data expected to "roll out over the course of 2017."
    ['(SNDX:NASDAQ)']
    Syros Pharmaceuticals (SYRS:NASDAQ) is an early-stage company, with a ~$250M market cap, that King considers a leader in gene regulation, with a concept called super-enhancers. The company both develops its "own internal candidates to some of the highly validated targets," as well as repurposes assets that may have been abandoned for indications like blood disorders. Syros might be "a quiet story the first half of 2017, but I think in the second half of 2017, we'll start to see some fireworks based on the data that will read out through key assets, SY-1365 and SY-1425."
    ['(SYRS:NASDAQ)']
    

    【讨论】:

      猜你喜欢
      • 2014-07-03
      • 2016-01-28
      • 2019-11-28
      • 2014-02-01
      • 2020-09-28
      • 1970-01-01
      • 1970-01-01
      • 2017-04-24
      • 1970-01-01
      相关资源
      最近更新 更多