【问题标题】:web scraping for data网页抓取数据
【发布时间】:2021-12-28 22:08:30
【问题描述】:

我得到一个

referenceError: 终端 A 未定义

在第 48 行。我不确定自己做错了什么。

writeStream.write(`Terminal A,Terminal B,Terminal C/D \n`);


var minutes = 1, timerInerval = minutes * 60 * 1000;


function TerminalOccupancyData() {

 
  request('https://www.laguardiaairport.com/to-from-airport/parking', (error, response, html) => {
    // Check there is no error
    if (!error && response.statusCode == 200) {
      // using cheerio library to load the website page html
      const $ = cheerio.load(html);

      $('.terminal-left').each((span, el) => {
        // Find the element using the class
        const terminalA = $(el)
          .find('.terminal-percentage')
          .text()
          .replace(/% Full/, '');

        const terminalB = $(el)
          .find('.terminal-percentage')
          .text()
          .replace(/% Full/, '');

        const terminalCD = $(el)
          .find('.terminal-percentage')
          .text()
          .replace(/% Full/, '');
      });

      console.log('\nTerminal Data scraped ... \n');
    }

  });

  // Export to file to upload to database
  writeStream.write(`${terminalA},${terminalB},${terminalCD} \n`);
}

setInterval(TerminalOccupancyData, timerInerval);

【问题讨论】:

    标签: node.js csv web-scraping export-to-csv


    【解决方案1】:

    您在这里遇到的问题是您的 writeStream.write(...) 与它所引用的变量不在同一范围内。由于您没有尝试从外部函数返回任何值,我相信解决您的问题的方法就是将该语句移到 .each() 回调函数中,如下所示:

    function TerminalOccupancyData() {
    
     
      request('https://www.laguardiaairport.com/to-from-airport/parking', (error, response, html) => {
        // Check there is no error
        if (!error && response.statusCode == 200) {
          // using cheerio library to load the website page html
          const $ = cheerio.load(html);
    
          $('.terminal-left').each((span, el) => {
            // Find the element using the class
            const terminalA = $(el)
              .find('.terminal-percentage')
              .text()
              .replace(/% Full/, '');
    
            const terminalB = $(el)
              .find('.terminal-percentage')
              .text()
              .replace(/% Full/, '');
    
            const terminalCD = $(el)
              .find('.terminal-percentage')
              .text()
              .replace(/% Full/, '');
    
              // Export to file to upload to database
              writeStream.write(`${terminalA},${terminalB},${terminalCD} \n`);
          });
    
          console.log('\nTerminal Data scraped ... \n');
    
        }
    
      });
    
      
    }
    
    setInterval(TerminalOccupancyData, timerInerval);
    

    【讨论】:

    • 谢谢。它解决了问题,但我没有将数据抓取到 csv
    猜你喜欢
    • 2021-10-15
    • 2021-08-29
    • 2020-06-18
    • 2020-09-26
    • 2013-11-05
    • 1970-01-01
    • 2020-05-10
    相关资源
    最近更新 更多