如何使用 beautifulsoup 选择特定表并打印其数据 [重复]

【问题标题】：how to select a particular table and print its data using beautifulsoup [duplicate]如何使用 beautifulsoup 选择特定表并打印其数据 [重复]
【发布时间】：2016-10-10 07:57:21
【问题描述】：

我有以下代码用于从名为 ssllabs,com 的网站获取结果

from bs4 import BeautifulSoup
import requests
req  = requests.get("https://www.ssllabs.com/ssltest/analyze.html?d=drtest.test.sentinelcloud.com")
data = req.text
soup = BeautifulSoup(data)
report_tables=soup.find_all('table',class_='reportTable')
print report_tables

这会返回以下表格：

现在我的数据在我指定的表格中。该表内部的结构看起来像 

<table class="reportTable">
   \n
   <thead>
      \n
      <tr>
         \n
         <td class="tableHead" colspan="3">Cipher Suites (SSL 3+ suites in server-preferred order; deprecated and SSL 2 suites at the end)</td>
         \n
      </tr>
      \n
   </thead>
   \n
   <tbody>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\n                                        (<code>0xc02f</code>)\n                                                            \xa0 <span class="greySmall"> ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\n                                        (<code>0xc030</code>)\n                                                            \xa0 <span class="greySmall"> ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_DHE_RSA_WITH_AES_128_GCM_SHA256\n                                        (<code>0x9e</code>)\n                                 \xa0\n                                    <span class="greySmall">\n<span title="p: 256, g: 1, Ys: 256">DH 2048 bits</span> \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_DHE_RSA_WITH_AES_256_GCM_SHA384\n                                        (<code>0x9f</code>)\n                                 \xa0\n                                    <span class="greySmall">\n<span title="p: 256, g: 1, Ys: 256">DH 2048 bits</span> \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256\n                                        (<code>0xc027</code>)\n                                                            \xa0 <span class="greySmall"> ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA\n                                        (<code>0xc013</code>)\n                                                            \xa0 <span class="greySmall"> ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384\n                                        (<code>0xc028</code>)\n                                                            \xa0 <span class="greySmall"> ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA\n                                        (<code>0xc014</code>)\n                                                            \xa0 <span class="greySmall"> ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_DHE_RSA_WITH_AES_128_CBC_SHA256\n                                        (<code>0x67</code>)\n                                 \xa0\n                                    <span class="greySmall">\n<span title="p: 256, g: 1, Ys: 256">DH 2048 bits</span> \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_DHE_RSA_WITH_AES_128_CBC_SHA\n                                        (<code>0x33</code>)\n                                 \xa0\n                                    <span class="greySmall">\n<span title="p: 256, g: 1, Ys: 256">DH 2048 bits</span> \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_DHE_RSA_WITH_AES_256_CBC_SHA256\n                                        (<code>0x6b</code>)\n                                 \xa0\n                                    <span class="greySmall">\n<span title="p: 256, g: 1, Ys: 256">DH 2048 bits</span> \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_DHE_RSA_WITH_AES_256_CBC_SHA\n                                        (<code>0x39</code>)\n                                 \xa0\n                                    <span class="greySmall">\n<span title="p: 256, g: 1, Ys: 256">DH 2048 bits</span> \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA\n                                        (<code>0xc012</code>)\n                                                            \xa0 <span class="greySmall"> ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">112</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_RSA_WITH_AES_128_GCM_SHA256\n                                        (<code>0x9c</code>)\n                                                                \n                    \n                </td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_RSA_WITH_AES_256_GCM_SHA384\n                                        (<code>0x9d</code>)\n                                                                \n                    \n                </td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_RSA_WITH_AES_128_CBC_SHA256\n                                        (<code>0x3c</code>)\n                                                                \n                    \n                </td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_RSA_WITH_AES_256_CBC_SHA256\n                                        (<code>0x3d</code>)\n                                                                \n                    \n                </td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_RSA_WITH_AES_128_CBC_SHA\n                                        (<code>0x2f</code>)\n                                                                \n                    \n                </td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_RSA_WITH_AES_256_CBC_SHA\n                                        (<code>0x35</code>)\n                                                                \n                    \n                </td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA\n                                        (<code>0x88</code>)\n                                 \xa0\n                                    <span class="greySmall">\n<span title="p: 256, g: 1, Ys: 256">DH 2048 bits</span> \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_RSA_WITH_CAMELLIA_256_CBC_SHA\n                                        (<code>0x84</code>)\n                                                                \n                    \n                </td>
         \n
         <td class="tableRight">256</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA\n                                        (<code>0x45</code>)\n                                 \xa0\n                                    <span class="greySmall">\n<span title="p: 256, g: 1, Ys: 256">DH 2048 bits</span> \xa0 FS</span>\n</td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_RSA_WITH_CAMELLIA_128_CBC_SHA\n                                        (<code>0x41</code>)\n                                                                \n                    \n                </td>
         \n
         <td class="tableRight">128</td>
         \n
      </tr>
      \n
      <tr class="tableRow">
         \n
         <td class="tableLeft">\n                                            TLS_RSA_WITH_3DES_EDE_CBC_SHA\n                                        (<code>0xa</code>)\n                                                                \n                    \n                </td>
         \n
         <td class="tableRight">112</td>
         \n
      </tr>
      \n
   </tbody>
   \n
</table>

我需要进入 'tbody' 并提取所有 tableLeft 值并将它们放在一个列表中。 我的问题：

1. How to select that particular reportTable at line 493 in picture.
2. How to extract the values (TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384.......................) and put in LIST

【问题讨论】：

  • 1. report_tables[4] 2. report_tables[4].find_all('td', class_='tableLeft')

标签： python beautifulsoup


【解决方案1】：

稍微扩展@furas 的评论，因为report_tables[4] 假设它始终是第 5 个表：

req = requests.get("https://www.ssllabs.com/ssltest/analyze.html?d=drtest.test.sentinelcloud.com")
data = req.text
soup = BeautifulSoup(data)

for found_table in soup.find_all('table', class_='reportTable'):
    if 'Cipher Suites' in found_table.get_text():
        values = found_table.find_all('td', class_='tableLeft')
        entries = []
        for row in values:
            entries.append(row.get_text())
        print entries

检查“密码套件”（尽管如果需要，您可以使用更完整的标题）应该可以帮助您更一致地获得正确的表格。

您可以简单地使用values 作为输出，但使用get_text() 可以帮助我们删除一些您可能不需要的html。 entries 将包含您需要的值，但您可能需要查看像 strip 这样的函数来清除结果中的空格。

产生的结果：

[u'\n                                            TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\n                                        (0xc02f)\n                                                            \xa0  ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS\n', u'\n                                            TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\n                                        (0xc030)\n                                                            \xa0  ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS\n', u'\n                                            TLS_DHE_RSA_WITH_AES_128_GCM_SHA256\n                                        (0x9e)\n                                 \xa0\n                                    \nDH 2048 bits \xa0 FS\n', u'\n                                            TLS_DHE_RSA_WITH_AES_256_GCM_SHA384\n                                        (0x9f)\n                                 \xa0\n                                    \nDH 2048 bits \xa0 FS\n', u'\n                                            TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256\n                                        (0xc027)\n                                                            \xa0  ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS\n', u'\n                                            TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA\n                                        (0xc013)\n                                                            \xa0  ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS\n', u'\n                                            TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384\n                                        (0xc028)\n                                                            \xa0  ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS\n', u'\n                                            TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA\n                                        (0xc014)\n                                                            \xa0  ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS\n', u'\n                                            TLS_DHE_RSA_WITH_AES_128_CBC_SHA256\n                                        (0x67)\n                                 \xa0\n                                    \nDH 2048 bits \xa0 FS\n', u'\n                                            TLS_DHE_RSA_WITH_AES_128_CBC_SHA\n                                        (0x33)\n                                 \xa0\n                                    \nDH 2048 bits \xa0 FS\n', u'\n                                            TLS_DHE_RSA_WITH_AES_256_CBC_SHA256\n                                        (0x6b)\n                                 \xa0\n                                    \nDH 2048 bits \xa0 FS\n', u'\n                                            TLS_DHE_RSA_WITH_AES_256_CBC_SHA\n                                        (0x39)\n                                 \xa0\n                                    \nDH 2048 bits \xa0 FS\n', u'\n                                            TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA\n                                        (0xc012)\n                                                            \xa0  ECDH secp256r1 (eq. 3072 bits RSA) \xa0 FS\n', u'\n                                            TLS_RSA_WITH_AES_128_GCM_SHA256\n                                        (0x9c)\n                                                                \n                    \n                ', u'\n                                            TLS_RSA_WITH_AES_256_GCM_SHA384\n                                        (0x9d)\n                                                                \n                    \n                ', u'\n                                            TLS_RSA_WITH_AES_128_CBC_SHA256\n                                        (0x3c)\n                                                                \n                    \n                ', u'\n                                            TLS_RSA_WITH_AES_256_CBC_SHA256\n                                        (0x3d)\n                                                                \n                    \n                ', u'\n                                            TLS_RSA_WITH_AES_128_CBC_SHA\n                                        (0x2f)\n                                                                \n                    \n                ', u'\n                                            TLS_RSA_WITH_AES_256_CBC_SHA\n                                        (0x35)\n                                                                \n                    \n                ', u'\n                                            TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA\n                                        (0x88)\n                                 \xa0\n                                    \nDH 2048 bits \xa0 FS\n', u'\n                                            TLS_RSA_WITH_CAMELLIA_256_CBC_SHA\n                                        (0x84)\n                                                                \n                    \n                ', u'\n                                            TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA\n                                        (0x45)\n                                 \xa0\n                                    \nDH 2048 bits \xa0 FS\n', u'\n                                            TLS_RSA_WITH_CAMELLIA_128_CBC_SHA\n                                        (0x41)\n                                                                \n                    \n                ', u'\n                                            TLS_RSA_WITH_3DES_EDE_CBC_SHA\n                                        (0xa)\n                                                                \n                    \n                ']

编辑：要根据@PadraicCunningham 的 cmets 扩展它，我们可以删除空格并返回第一个值，如下所示：

for found_table in soup.find_all('table', class_='reportTable'):
    if 'Cipher Suites' in found_table.get_text():
        vals = [td.text.split()[0] for td in found_table.select("td.tableLeft")]
        print vals
        break

【讨论】：

  • @PadraicCunningham 我认为他的意思是第四个索引。我基本上做了与 furas 相同的操作，并使用了 report_tables[4]，然后使用了这样的 for 循环：for td in report_tables.find_all('td', class_='tableLeft'): for line in td: my_data.append(line)。
  • @Kittystone, table = soup.find_all('table',class_='reportTable', limit=5)[-1][td.text.split()[0] for td in table.select("td.tableLeft")] 假设你只想要 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 等等。
  • 您将所有内容放在一行中，列表 comp 应该在它自己的行中，这就是保存数据的内容，即 data = [td.text.split()[0] for td in table.select("td.tableLeft")]
  • 知道了，非常感谢:)
  • @PadraicCunningham 为延迟回复道歉 - 你在每个 cmets 中都是绝对正确的。我已尝试将它们合并到答案中，谢谢。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2021-09-24
  • 1970-01-01
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode