使用 python 抓取具有不变 URL 的多个表的最佳资源答案

【问题标题】：Best resource for Scraping multiple tables with unchanging URL using python使用 python 抓取具有不变 URL 的多个表的最佳资源
【发布时间】：2017-11-15 14:01:37
【问题描述】：

我想在有下一个链接的网页上抓取表格，但是当您单击这些下一个按钮时，URL 不会改变。学习使用 python (BeautifulSoup、Requests) 抓取这些类型的表的最佳/最简单的资源是什么？

例如，我如何抓取this page from Boone County Sheriff's Department 上的表格。

【问题讨论】：

标签： python web-scraping beautifulsoup

【解决方案1】：

您将需要使用 Selenium 而不是请求。这样您就可以与基于 javascript 的操作进行交互。

代码：

import re
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

chromeOptions = Options()
chromeOptions.add_argument("--headless")
driver = webdriver.Chrome(chrome_options=chromeOptions)
wait = WebDriverWait(driver, 30)
driver.get('https://report.boonecountymo.org/mrcjava/servlet/SH01_MP.I00290s')

soup = BeautifulSoup(driver.page_source, 'html.parser')
page_numbers = int(str(soup.find('a', string=re.compile('Page 1 of '), href='#').text).split('Page 1 of ')[1])
print(page_numbers)

tables = []

for page in range(1, page_numbers + 1):
    if page == 1:
        table = soup.find('tbody', id='mrc_main_table')
        #print(table)
        tables.append(table)
    else:
        wait.until(EC.visibility_of_element_located((By.XPATH, '//a[@class="page-button"][@data-pagetype="btn_next"]')))
        driver.find_element_by_xpath('//a[@class="page-button"][@data-pagetype="btn_next"]').click()
        soup = BeautifulSoup(driver.page_source, 'html.parser')
        table = soup.find('tbody', id='mrc_main_table')
        #print(table)
        tables.append(table)

print(tables)

输出：

5
[<tbody class="stripe" id="mrc_main_table">
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">AARON</td>
<td class="one td_left" data-th="First Name">KEVIN</td>
<td class="one td_left" data-th="Middle Name">PHILLIP</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">23</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=66343&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">ADAM</td>
<td class="two td_left" data-th="First Name">OMER</td>
<td class="two td_left" data-th="Middle Name">SIRAJ</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">B</td>
<td class="two td_right" data-th="Age">29</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=77695&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">AKERS</td>
<td class="one td_left" data-th="First Name">BRANDEN</td>
<td class="one td_left" data-th="Middle Name">MICHAEL</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">W</td>
<td class="one td_right" data-th="Age">32</td>
<td class="one td_left" data-th="City">MOKANE</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=51443&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">AKERS</td>
<td class="two td_left" data-th="First Name">SYDNEY</td>
<td class="two td_left" data-th="Middle Name">RAE</td>
<td class="two td_left" data-th="Sex">F</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">20</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=76028&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">ALLEN</td>
<td class="one td_left" data-th="First Name">WILLIAM</td>
<td class="one td_left" data-th="Middle Name">LAMAR</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">55</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=19022&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">ALVARADO</td>
<td class="two td_left" data-th="First Name">JACOVO</td>
<td class="two td_left" data-th="Middle Name"> </td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">H</td>
<td class="two td_right" data-th="Age">34</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=79782&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">ANDERSON</td>
<td class="one td_left" data-th="First Name">RICHARD</td>
<td class="one td_left" data-th="Middle Name">KENNETH</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">24</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=66544&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">AVALOS-AVALOS</td>
<td class="two td_left" data-th="First Name">JOSE</td>
<td class="two td_left" data-th="Middle Name"> </td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">H</td>
<td class="two td_right" data-th="Age">19</td>
<td class="two td_left" data-th="City">ST.ANN</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=78424&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">AVILES</td>
<td class="one td_left" data-th="First Name">LUCIANO</td>
<td class="one td_left" data-th="Middle Name">ESPINOZA</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">H</td>
<td class="one td_right" data-th="Age">21</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=79931&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">BAILEY</td>
<td class="two td_left" data-th="First Name">JAMES</td>
<td class="two td_left" data-th="Middle Name">LEROY</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">50</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=18934&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">BARNEY</td>
<td class="one td_left" data-th="First Name">FELSON</td>
<td class="one td_left" data-th="Middle Name">DEVONE</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">40</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=4285&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">BARTON</td>
<td class="two td_left" data-th="First Name">JAMES</td>
<td class="two td_left" data-th="Middle Name">EDWARD</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">63</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=76341&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">BELL</td>
<td class="one td_left" data-th="First Name">COLIN</td>
<td class="one td_left" data-th="Middle Name">KEITH</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">W</td>
<td class="one td_right" data-th="Age">35</td>
<td class="one td_left" data-th="City">HALLSVILLE</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=33000&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">BENEDETTI</td>
<td class="two td_left" data-th="First Name">FRANK</td>
<td class="two td_left" data-th="Middle Name">DOMINICK</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">29</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=79913&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">BENNETT</td>
<td class="one td_left" data-th="First Name">ROBERT</td>
<td class="one td_left" data-th="Middle Name">DAVID</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">39</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=33666&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">BERARDI</td>
<td class="two td_left" data-th="First Name">DEBORAH</td>
<td class="two td_left" data-th="Middle Name">EVA</td>
<td class="two td_left" data-th="Sex">F</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">56</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=78005&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">BLAIR</td>
<td class="one td_left" data-th="First Name">RYAN</td>
<td class="one td_left" data-th="Middle Name">WADE</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">W</td>
<td class="one td_right" data-th="Age">26</td>
<td class="one td_left" data-th="City">MEXICO</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=73590&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">BOND</td>
<td class="two td_left" data-th="First Name">SOPHIA</td>
<td class="two td_left" data-th="Middle Name">LYNN</td>
<td class="two td_left" data-th="Sex">F</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">23</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=67252&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">BOONE</td>
<td class="one td_left" data-th="First Name">NIKKI</td>
<td class="one td_left" data-th="Middle Name">MARIE</td>
<td class="one td_left" data-th="Sex">F</td>
<td class="one td_left" data-th="Race">W</td>
<td class="one td_right" data-th="Age">33</td>
<td class="one td_left" data-th="City">COLUMBIALE</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=39306&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">BOSS</td>
<td class="two td_left" data-th="First Name">JAFARI</td>
<td class="two td_left" data-th="Middle Name">RASHIDI</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">B</td>
<td class="two td_right" data-th="Age">34</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=60725&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">BOYDSTON</td>
<td class="one td_left" data-th="First Name">LAVERNE</td>
<td class="one td_left" data-th="Middle Name">KEHAULANI</td>
<td class="one td_left" data-th="Sex">F</td>
<td class="one td_left" data-th="Race">A</td>
<td class="one td_right" data-th="Age">59</td>
<td class="one td_left" data-th="City">MOBERLY</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=79939&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">BRILL</td>
<td class="two td_left" data-th="First Name">BRANDON</td>
<td class="two td_left" data-th="Middle Name">NICHOLAS</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">26</td>
<td class="two td_left" data-th="City">CLARK</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=60089&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">BROWN</td>
<td class="one td_left" data-th="First Name">THYRUS</td>
<td class="one td_left" data-th="Middle Name">MONTEZ</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">48</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=45497&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">BRYANT</td>
<td class="two td_left" data-th="First Name">CHESTER</td>
<td class="two td_left" data-th="Middle Name">LEROY</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">B</td>
<td class="two td_right" data-th="Age">53</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=10513&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
................
ETC...
................

<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">WILLIAMS</td>
<td class="one td_left" data-th="First Name">SYLVESTER</td>
<td class="one td_left" data-th="Middle Name">EARL</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">50</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=8656&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">WINTER</td>
<td class="two td_left" data-th="First Name">PAIGE</td>
<td class="two td_left" data-th="Middle Name">LYNN</td>
<td class="two td_left" data-th="Sex">F</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">20</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=75775&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">WISLEY</td>
<td class="one td_left" data-th="First Name">JORDAN</td>
<td class="one td_left" data-th="Middle Name">MONTEZ</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">27</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=58487&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">WOODS</td>
<td class="two td_left" data-th="First Name">DANIEL</td>
<td class="two td_left" data-th="Middle Name">SCOTT</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">33</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=39227&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">WOODWARD</td>
<td class="one td_left" data-th="First Name">ELAINIA</td>
<td class="one td_left" data-th="Middle Name">MARGURITE</td>
<td class="one td_left" data-th="Sex">F</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">19</td>
<td class="one td_left" data-th="City">TOPEKA</td>
<td class="one td_left" data-th="State">KS</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=79008&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">WYATT</td>
<td class="two td_left" data-th="First Name">CHASE</td>
<td class="two td_left" data-th="Middle Name">ROY</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">W</td>
<td class="two td_right" data-th="Age">31</td>
<td class="two td_left" data-th="City">CENTRALIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=59360&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">YEARWOOD</td>
<td class="one td_left" data-th="First Name">JAMES</td>
<td class="one td_left" data-th="Middle Name">CHRISTIAN</td>
<td class="one td_left" data-th="Sex">M</td>
<td class="one td_left" data-th="Race">B</td>
<td class="one td_right" data-th="Age">19</td>
<td class="one td_left" data-th="City">COLUMBIA</td>
<td class="one td_left" data-th="State">MO</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=77805&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="even">
<td class="two td_left" data-th="Last Name">ZARHOUNI</td>
<td class="two td_left" data-th="First Name">DRISS</td>
<td class="two td_left" data-th="Middle Name">BURCH</td>
<td class="two td_left" data-th="Sex">M</td>
<td class="two td_left" data-th="Race">H</td>
<td class="two td_right" data-th="Age">53</td>
<td class="two td_left" data-th="City">COLUMBIA</td>
<td class="two td_left" data-th="State">MO</td>
<td class="two td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=33263&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="odd">
<td class="one td_left" data-th="Last Name">ZWICKY</td>
<td class="one td_left" data-th="First Name">CARRIE</td>
<td class="one td_left" data-th="Middle Name">ANN</td>
<td class="one td_left" data-th="Sex">F</td>
<td class="one td_left" data-th="Race">W</td>
<td class="one td_right" data-th="Age">45</td>
<td class="one td_left" data-th="City">GALESBURG</td>
<td class="one td_left" data-th="State">IL</td>
<td class="one td_left" data-th="">
<a class="_lookup btn btn-primary" height="600" href="SH01_MP.I00500s?PERKEP=79930&amp;hover_redir=&amp;height=600&amp;width=950" linkedtype="I" mrc="returndata" target="_lookup" width="860"><i class="fa fa-large fa-fw fa-list-alt"> </i>Details</a>
</td>
</tr>
<!-- -->
<tr class="tr-paging">
<th class="th_left paging_row bottomPaging" colspan="93">  Detainee Count: 235
                            <div class="pagination pagination-right fixed-pagination" style="margin-left: 2em;">
<ul>
<li>
<a class="page-button" data-pagetype="btn_top" href="javascript:void(0)"><i class="fa fa-angle-double-left fa-large"> </i></a>
</li>
<li>
<a class="page-button" data-pagetype="btn_prev" href="javascript:void(0)"><i class="fa fa-angle-left fa-large"> </i></a>
</li>
<li class="disabled">
<a href="#">Page 5 of 5</a>
</li>
<li>
<a class="" data-pagetype="btn_next" href="javascript:void(0)"><i class="fa fa-angle-right fa-large"> </i></a>
</li>
<li>
<a class="" data-pagetype="btn_btmm" href="javascript:void(0)"><i class="fa fa-angle-double-right fa-large"> </i></a>
</li>
</ul>
</div>
</th>
</tr>
</tbody>]

【讨论】：

脚本会根据 URL 中的页面数量动态解析所有页面。我提供的输出故意缺少结果，因为我可以在答案中使用多少个字符。