从网站获取完整的 href 属性列表 [重复]答案

【问题标题】：get complete list of href attribute from website [duplicate]从网站获取完整的 href 属性列表 [重复]
【发布时间】：2017-09-14 07:58:22
【问题描述】：

我想从链接结构如下的网站获取为 YouTube 播放列表生成的所有 URLs 的完整列表：

<table class="table" id="ListVideo">
    <tbody>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
        .
        . and so on 
        .
        .
    </tbody>
</table>

我可以使用

从a tag 获取单个href

console.log($("#ListVideo td div a").attr("href"))

这给了我链接 https://redirector.googlevideo.com/videoplayback...0Brief%20Overview%20Of%20Android我要但是当我尝试使用以下函数遍历所有链接时

$("#ListVideo td div a").each(()=> {console.log($(this).attr("href"))})

相反，我得到了未定义。这是JSfiddle 链接，其中包含我想从中获取链接的完整页面源。有人可以告诉我哪里出错了。谢谢

【问题讨论】：

那里的解决方案与我尝试过的相同，但不起作用
可能是因为您试图从 div 中获取 href 属性（不会有）
抱歉忘了在这里添加标签，虽然我用它来获取所有链接，更新了我的问题
在这种情况下，您可能无法在 each 中执行类似的匿名函数

标签： javascript jquery html web-scraping

【解决方案1】：

使用正确的方法来获得正确的$(this) - 所以在这种情况下不是箭头函数。

var list = [];
$("#ListVideo td div a").each(function() {
  list.push($(this).attr("href"))
});
console.log(list);

使用箭头，您不需要使用this，如

Using jQuery $(this) with ES6 Arrow Functions (lexical this binding)

var list1 = [];
$("#ListVideo td div a").each((index, element) => {
  list1.push($(element).attr("href"));
});
console.log(list1);

https://jsfiddle.net/mplungjan/yr39tmqn/

【讨论】：

感谢您提供有关 $(this) 和 es6 箭头函数的信息

【解决方案2】：

首先，如何根据属性选择元素，例如$([href])？

其次，()=> {console.log($(this).attr("href"))} 未定义为箭头功能的日志具有不同的this。简而言之，箭头函数不会创建自己的this 上下文，因此this 的原始含义来自封闭的上下文。你可以阅读更多关于它的信息here。使用经典的 function 对你有用。

【讨论】：

【解决方案3】：

这是您的updated fiddle。您可以看到确切登出到控制台的链接数。

console.log($("#ListVideo td div a").attr("href"));
var hrefArray = [];
$("#ListVideo td div a").each(function(){
    console.log($(this).attr("href"));
  hrefArray.push($(this).attr("href"));
});

console.log(hrefArray.length);

【讨论】：

没有区别。在发布我的之前看不到您的答案。

【解决方案4】：

这是一个工作示例，纯 JavaScript 代码。我们使用querySelectorAll 在我们的.table 类中查找所有anchors。

var anchors = document.querySelectorAll('.table a');


// pure js
for (i = 0; i < anchors.length; i++) {
  var anchor = anchors[i];
  
  // You can access href attribute via anchor.href
  console.log(anchor.href);
}

// using forEach, we must use Object.keys 
Object.keys(anchors).forEach(function(key) {
    // console.log(anchors[key].href);  // href
});


// using map, we must use Object.keys 
var listOfAnchors = Object.keys(anchors).map(function(key) {
  return anchors[key].href;
});

// Array of anchors inside our table
// console.log(listOfAnchors);

<table class="table" id="ListVideo">
    <tbody>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
    </tbody>
</table>

【讨论】：

为什么要重写脚本呢？使用正确的this 时有效