如何通过 puppeteer 进行网页抓取

Question

如果我转到https://investor.vanguard.com/mutual-funds/profile/VMMXX 并从控制台执行document.querySelector("[data-ng-if='productSummaryTitle']").innerText，我会得到我所期望的：Product summary。

但是当我尝试对puppeteer 做同样的事情时，我得到了UnhandledPromiseRejectionWarning: Error: Evaluation failed: TypeError: Cannot read property 'innerText' of null at __puppeteer_evaluation_script__:3:83。我错过了什么？

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch({ headless: false })
    const page = await browser.newPage()
    await page.goto('https://investor.vanguard.com/mutual-funds/profile/VMMXX')

    const result = await page.evaluate(() => {
        let myText = document.querySelector("[data-ng-if='productSummaryTitle']").innerText
        return {
            myText
        }
    })

    console.log(result)

    browser.close()
})()

Answer 1

你可以先等待那个选择器

const element = await page.waitForSelector('[data-ng-if='productSummaryTitle']');
const text = await element.evaluate(el => el.innerText);

Answer 2

const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://investor.vanguard.com/mutual-funds/profile/VMMXX');
const element = await page.waitForSelector("[data-ng-if='productSummaryTitle']");
const text = await element.evaluate(el => el.innerText);
console.log(text);
await browser.close();
})();