在 MapReduce MongoDB 中按键对数据进行分组

Question

我正在MongoDB中尝试使用MapReduce程序来寻找共同的朋友，我在mongoDB中对key进行排序后得到了以下数据

{"user" : " Hari","friend" : "Shiva",
 "friendList": ["Hanks"," Tom"," Karma"," Hari"," Dinesh"]}


 {"user" : "Hari","friend" : " Shiva",
  "friendList" : ["Karma"," Tom"," Ram"," Bindu"," Shiva",
                   " Kishna"," Bikash"," Bakshi"," Dinesh"]}

现在我想将这些具有相同键的数据集分组到一个组中，在将键值对发送到减速器之前，在 map 函数中使用 Javascript，我该如何对数据进行分组？例如我想要输出像

{"user" : " Hari","friend" : "Shiva",
 "friendList": ["Hanks"," Tom"," Karma"," Hari"," Dinesh"],["Karma"," Tom"," Ram"," Bindu"," Shiva"," Kishna"," Bikash"," Bakshi"," Dinesh"]}

Answer 1

您可以将两条记录的friendlist 数组合并为一个数组，以创建如下对象：

   {
  "_id": {
    "user": " Hari",
    "friend": "Shiva"
  },
  "value": {
    "friendList": [
      "Hanks",
      " Tom",
      " Karma",
      " Hari",
      " Dinesh",
      "Karma",
      " Tom",
      " Ram",
      " Bindu",
      " Shiva",
      " Kishna",
      " Bikash",
      " Bakshi",
      " Dinesh"
    ]
  }
}

查看https://jsfiddle.net/b6hxswvk/1/ 的代码来创建这个单一对象

如果您希望 friendlist 成为二维数组，即像这样：

{
  "_id": {
    "user": " Hari",
    "friend": "Shiva"
  },
  "value": {
    "friendList": [
      [
        "Hanks",
        " Tom",
        " Karma",
        " Hari",
        " Dinesh"
      ],
      [
        "Karma",
        " Tom",
        " Ram",
        " Bindu",
        " Shiva",
        " Kishna",
        " Bikash",
        " Bakshi",
        " Dinesh"
      ]
    ]
  }
}

您可以使用https://jsfiddle.net/b6hxswvk/2/的代码

Answer 2

您可以简单地执行aggregation，您可以根据user 和friend 字段执行$group。

db.collection.aggregate([
{$group:{
_id:{
       user:'$user',
       friend:'$friend'
    },
    friendList:{$push:'$friendList'}
}},

// project the fields as your wish
{$project:{
    user:'$_id.user',
    friend:'$_id.friend',
    friendList:'$friendList'
}}
])

希望这个聚合管道可以返回您预期的结果

Answer 3

朋友，如果 map reduce 将通过对同一键的值进行分组并将其作为 key,list[values] 提供给 reduce 来执行它，为什么你要为同一个键的数据值分组？

我强烈建议您在 reducer 而不是 Map 中执行分组任务。其背后的主要原因是由于map任务逐条读取并执行收集操作，识别相同键组的负担由算法承担，如何设计具有分组值的输出可以由我们在reduce中负责逻辑

您可以将减速器的输出用于进一步处理。

输入：

{"_id" : {"user" : " Hari","friend" : "Shiva"},
 "value" : {"friendList": ["Hanks"," Tom"," Karma"," Hari"," Dinesh"]}}


 {"_id" : {"user" : "Hari","friend" : " Shiva"},
  "value" : {"friendList" : ["Karma"," Tom"," Ram"," Bindu"," Shiva",
                             " Kishna"," Bikash"," Bakshi"," Dinesh"]}}

Mapreduce 代码：

var mapper = function () {
    var key = {"user" : this.user, "friend" : this.friend};
    emit(key, {"value":{"friendList":this.friendList}});
};

var reducer = function(key, value){
var combinedfriendList = {"friendList":[]};

    for (var i in values) {
        var inter = values[i];
        for (var j in inter.friendList) {
            combinedfriendList.friendList.push(inter.friendList[j]);
        }
    }
return {"_id": {"user":key.user, "friend": key.friend}, "value":combinedfriendList};
};

预期输出：

{"_id" : {"user" : " Hari","friend" : "Shiva"},
 "value" : {"friendList": ["Hanks"," Tom"," Karma"," Hari"," Dinesh","Karma"," Tom"," Ram"," Bindu"," Shiva"," Kishna"," Bikash"," Bakshi"," Dinesh"]}}

希望这对您有所帮助。您可以在您的环境中对其进行测试（如果需要进行更改）并分享您的反馈。