从表中提取多个“td”类

Question

我有一个网站，上面有很多表，我正在循环访问它们并从所有表中获取数据并将其导出到 csv 文件。我遇到的问题是每个表都有多个“td”类，我只想要其中几个，但我不知道如何将它们缩小到我想要的。 当我运行我当前的代码时，我得到了我不想要的表格的额外部分，这使我的 csv 文件非常混乱。 这是包含我想要的信息的页面的 html 代码的 sn-p。

<tr class="table_title">
  <td class="sportPicksBorderL2 tanBg fourleft regular" nowrap="nowrap">
    607 <a class="black" href="/college-basketball/teams/team-page.cfm/team/oakland">OAKLND</a>
  </td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">&nbsp;136.5&nbsp;</td>

  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">28</td>

  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">32</td>


  <td class="sportPicksBorderL2 tanBg zerocenter sub_title_red" nowrap="nowrap">60</td>

  <td class="sportPicksBorderR2 tanBg zerocenter regular" nowrap="nowrap" width="100">Cover: +1.5&nbsp;</td>

</tr>
<tr class="table_title">
  <td class="sportPicksBorderL2 tanBg fourleft regular" nowrap="nowrap">
    608 <a class="black" href="/college-basketball/teams/team-page.cfm/team/youngstown-state">YOUNG</a> <span class="sub_title_red">«</span>
  </td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">&nbsp;-2.5&nbsp;</td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">24</td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">37</td>

  <td class="sportPicksBorderL2 tanBg zerocenter sub_title_red" nowrap="nowrap">61</td>

  <td class="sportPicksBorderR2 tanBg zerocenter regular" nowrap="nowrap" width="100">Under: 121&nbsp;</td>

</tr>

这是我用来提取数据的方法

import bs4 as bs
import urllib.request
from requests import get
import pandas as pd
from csv import writer
url = 'https://www.vegasinsider.com/college-basketball/scoreboard/scores.cfm/game_date/01-12-2020'
source = urllib.request.urlopen(url)
soup = bs.BeautifulSoup(source, 'lxml')
tables = soup.find_all('table', attrs={'class': 'sportPicksBorder'})
filename = 'C:\\Users\\mylocation\\Documents\\tables.csv'
with open(filename, 'w')as f:
    data = []
    csv_writer = writer(f)
    for table in tables:
        rows = table.find_all('tr')
        for row in rows:
            columns = row.find_all('td')
            csv_writer.writerow([column.text.strip() for column in columns])

我想要的是 class="black"/a 这有我的团队名称，旁边没有红色箭头 class=sportPicksBorderL2 class=sportPicksBorderR2

我的原始代码为每个表返回这 4 行。

Final Score Score                                                   
Teams   Odds    1   2   T   ATS                                 
847 ARIZNA  -4  31  34  65  Over: 147                                   
848 OREGST «   143.5   31  51  82  Cover: +21

我想避免的是。

ARIZNA  -4  31  34  65  Over: 147                                   
OREGST 143.5 31 51  82  Cover: +21

我想从我正在抓取的内容中删除 «、团队名称前的数字以及前 2 个表格行，但我仍然是网络抓取和 python 的新手，我是否发现很难弄清楚.如果有人有好的培训资源，我会全力以赴。

Answer 1

你想要以下：

var blacks = document.getElementsByClassName('black');

var blacks 将是一个数组，您可以按如下方式访问每个元素：

for(i = 0; i< blacks.length;i++){
    var black = blacks[i];
}

Answer 2

另一种解决方案。

from simplified_scrapy.spider import SimplifiedDoc 
html='''
<table class="sportPicksBorder">
 <tr class="table_title">
  <td class="sportPicksBorderL2 tanBg fourleft regular" nowrap="nowrap">
    607 <a class="black" href="/college-basketball/teams/team-page.cfm/team/oakland">OAKLND</a>
  </td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">&nbsp;136.5&nbsp;</td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">28</td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">32</td>
  <td class="sportPicksBorderL2 tanBg zerocenter sub_title_red" nowrap="nowrap">60</td>
  <td class="sportPicksBorderR2 tanBg zerocenter regular" nowrap="nowrap" width="100">Cover: +1.5&nbsp;</td>
</tr>
<tr class="table_title">
  <td class="sportPicksBorderL2 tanBg fourleft regular" nowrap="nowrap">
    608 <a class="black" href="/college-basketball/teams/team-page.cfm/team/youngstown-state">YOUNG</a> <span class="sub_title_red">«</span>
  </td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">&nbsp;-2.5&nbsp;</td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">24</td>
  <td class="sportPicksBorderL2 tanBg zerocenter regular" nowrap="nowrap">37</td>
  <td class="sportPicksBorderL2 tanBg zerocenter sub_title_red" nowrap="nowrap">61</td>
  <td class="sportPicksBorderR2 tanBg zerocenter regular" nowrap="nowrap" width="100">Under: 121&nbsp;</td>
</tr>
</table>
'''
doc = SimplifiedDoc(html) # create doc
tables = doc.getElementsByClass('sportPicksBorder')
# If you only want to include 'td' of 'a' and remove the arrow at the back, you can do this
for table in tables:
    rows = table.trs
    for row in rows:
        column = row.td # Get the first td
        column.removeElement('span') # Delete that arrow
        print (column.text)
        # You can do the same
        print (column.firstText(),column.a.text)

结果：

607 OAKLND
607 OAKLND
608 YOUNG
608 YOUNG

这里有更多例子：https://github.com/yiyedata/simplified-scrapy-demo/blob/master/doc_examples/

Answer 3

我发现在我的列循环中，我需要以这种方式格式化我的代码，以便只找到我想要的特定“td”

columns = row.find_all(class_=["black", "sportPicksBorderL2 tanBg zerocenter regular", "sportPicksBorderL2 tanBg zerocenter sub_title_red", "sportPicksBorderR2 tanBg zerocenter regular"])

使用 find_all 查找所有类，然后使用 class_ 查看所有这些类以找到想要的类，我需要用逗号分隔它们，然后按照我希望它们显示在我的 csv 文件中的顺序放置它们需要做的。