如何将来自 for 循环的单独 json 对象输出连接到数组中？

Question

我正在使用 CasperJS 抓取网站，其中一项任务涉及爬取由 for 循环计数器设置的 url。网址是这样的

www.example.com/page/no=

其中 no 是由 for 循环计数器设置的 0-10 之间的任意数字。然后刮板遍历所有页面，将数据刮到 JSON 对象中并重复直到 no=10。

我试图获取的数据存储在每个页面中的离散组中——我希望通过连接每个页面的所有抓取输出来使用单个 JSON 对象。

假设 Page1 有 Expense 1，我得到的对象是 {expense1}，Page 2 有 Expense 2，我得到的对象是 {expense2}。我想要的是在抓取结束时有一个 JSON，如下所示：

    scrapedData = {
       "expense1": expense1,
       "expense2": expense2,
     }

我遇到的麻烦是将所有 JSON 对象连接到一个数组中。

我初始化了一个空数组，然后每个对象都被推送到数组中。 我尝试检查 for 循环中的迭代器 i 是否等于 10，然后打印出 JSON 对象，但这似乎不起作用。我抬头一看，似乎 Object spread 是一个选项，但我不确定在这种情况下如何使用它。

任何指针都会有所帮助。我应该使用 map 之类的数组函数吗？

casper.then(function(){    

   var url = "https:example.net/secure/SaFinShow?url=";    
    //We create a for loop to go open the urls

    for (i=0; i<11; i++){

      this.thenOpen(url+ i, function(response){

          expense_amount = this.fetchText("td[headers='amount']");

          Date = this.fetchText("td[headers='Date']");

          Location = this.fetchText("td[headers='zipcode']");

          id = this.fetchText("td[headers='id']");


          singleExpense = {

              "Expense_Amount": expense_amount,
              "Date": Date,
              "Location": Location,
              "id": id
            };

          if (i ===10){
            expenseArray.push(JSON.stringify(singleExpense, null, 2))
            this.echo(expenseArray);
          }
      });

    };
});

Answer 1

以您的示例并对其进行扩展，您应该能够执行以下操作：

// Initialize empty object to hold all of the expenses
var scrapedData = {};

casper.then(function(){    

   var url = "https:example.net/secure/SaFinShow?url=";    
    //We create a for loop to go open the urls

    for (i=0; i<11; i++){

      this.thenOpen(url+ i, function(response){

          expense_amount = this.fetchText("td[headers='amount']");

          Date = this.fetchText("td[headers='Date']");

          Location = this.fetchText("td[headers='zipcode']");

          id = this.fetchText("td[headers='id']");


          singleExpense = {

              "Expense_Amount": expense_amount,
              "Date": Date,
              "Location": Location,
              "id": id
            };
          // As we loop over each of the expenses add them to the object containing all of them
          scrapedData['expense'+i] = singleExpense;
      });

    };
});

运行后，scrapedData 变量应为以下形式：

scrapedData = {
  "expense1": expense1,
  "expense2": expense2
}

更新代码

上述代码的一个问题是，在循环费用时，在 for 循环中，变量应该是本地的。变量名也不应该是 Date 和 Location，因为它们是 JavaScript 中的内置名称。

// Initialize empty object to hold all of the expenses
var scrapedData = {};

casper.then(function(){    

   var url = "https:example.net/secure/SaFinShow?url=";    
    //We create a for loop to go open the urls

    for (i=0; i<11; i++){

      this.thenOpen(url+ i, function(response){
          // Create our local variables to store data for this particular
          // expense data
          var expense_amount = this.fetchText("td[headers='amount']");

          // Don't use `Date` it is a JS built-in name
          var date = this.fetchText("td[headers='Date']");
          // Don't use `Location` it is a JS built-in name
          var location = this.fetchText("td[headers='zipcode']");

          var id = this.fetchText("td[headers='id']");


          singleExpense = {

              "Expense_Amount": expense_amount,
              "Date": date,
              "Location": location,
              "id": id
            };

          // As we loop over each of the expenses add them to the object containing all of them
          scrapedData['expense'+i] = singleExpense;
      });

    };
});