将 Javascript Web 抓取导出到 Excel

Question

我有一个脚本，我正尝试在大学中使用它来访问有关组织的信息。代码运行，我得到了公司描述的输出，但我不知道如何将它导出到 Excel。有人有什么想法吗？

以下代码摘录：

async function scrapeProduct(url) {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto(url);

    const [el] = await page.$x('//*[@id="hits"]/div[1]');
    const txt = await el.getProperty('textContent');
    const rawTxt = await txt.jsonValue();

    console.log({rawTxt});

}

scrapeProduct('https://www.ibisworld.com/search/default.aspx?st=International%Game%Technology%Plc')

编辑：

我已将代码更新为以下内容：

const puppeteer = require('puppeteer');
const fs = require('fs');
const stringify = require('csv-stringify');

async function scrapeProduct(url) {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto(url);

    const [el] = await page.$x('//*[@id="hits"]/div[1]');
    const txt = await el.getProperty('textContent');
    const rawTxt = await txt.jsonValue();

    stringify(rawTxt, function(err, output){
        fs.writeFile("output.csv", output);
      });

    console.log({rawTxt});

}

scrapeProduct('https://www.ibisworld.com/search/default.aspx?st=International%Game%Technology%Plc')

但现在收到以下错误消息：

C:\Users\User\scrape>node scrapers.js
(node:32096) UnhandledPromiseRejectionWarning: Error: Invalid argument: got "IGT (Australia) Pty Limited IGT (Australia) Pty Limited is a foreign owned private company, deriving revenue from the supply and servicing of gaming machines and related products. The company employs approximately 300 people, operates in Australia and New Zealand, and is administered from its head office in Macquarie Park, New South Wales. IGT is a wholly owned subsidiary of the UK-based gaming company, International Game Technologies PLC.Subsidiaries: IGT (Australia) Pty Limited, International Game Technology (NZ) LtdKey personnel: Andrew Neagle, Isaac Ai, Claudio Demolli, Phil Osborne, John van Waard, Nigel Turner" at index 0
    at stringify (C:\Users\User\scrape\node_modules\csv-stringify\lib\index.js:480:13)
    at scrapeProduct (C:\Users\User\scrape\scrapers.js:15:5)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:32096) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:32096) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Answer 1

我建议您使用 csv-stringify 之类的模块将 JSON 输出转换为 CSV，然后使用 fs 将其保存到文件中。如果您想避免require，手动将 JSON 转换为 CSV 也是可行的。

const fs = require('fs');
const stringify = require('csv-stringify');

...

stringify(rawTxt, function(err, output){
  fs.writeFile("output.csv", output);
});

编辑： 为了解决您之前的编辑问题，我想补充一点，您提供给csv-stringify 的数据必须是二维数组或列表列表。这是你有一个网格，比如一个 Excel 电子表格，它有行和列。例如：

二维数组输入：[ [a,b,c], [d,e,f], [g,h,i] ]
将由csv-stringify 转换为：a,b,c\n,d,e,f\n,g,h,i\

当在 Excel 等 CSV 查看器中打开时，会如下所示：

|a|b|c|
-------
|d|e|f|
-------
|g|h|i|

当以 CSV 格式快速布局时