MongoDb Pipeline Aggregation 排序子子文档答案

【问题标题】：MongoDb Pipeline Aggregation sorting sub sub documentsMongoDb Pipeline Aggregation 排序子子文档
【发布时间】：2015-07-23 07:23:58
【问题描述】：

我在尝试使用 MongooseJs 在 Mongodb 中按嵌套数组排序时遇到了一个小问题。

a) 产品包含任务，每个任务都有子任务。
b) 任务有顺序，每个子任务也有顺序（task.order & task.subtask.order）。

这是一个示例产品文档：

db.products.find({_id: ObjectId("554a13d4b692088a38f01f3b")})

结果：

{
"_id" : ObjectId("554a13d4b692088a38f01f3b"),
"title" : "product title",
"order" : 3,
"description" : "Description here ",
"status" : "live",
"tasks" : [ 
    {
        "title" : "task 1",
        "description" : "task 1 desc",
        "order" : 10,
        "_id" : ObjectId("554a13d4b692088a38f01f3a"),
        "status" : "live",
        "subTasks" : [ 
            {
                "title" : "task 1 sub 1",
                "content" : "aaa",
                "order" : -2,
                "_id" : ObjectId("554a13d4b692088a38f01f5a"),
                "status" : "live"
            }, 
            {
                "title" : "task 1 sub 2",
                "content" : "aaa",
                "order" : 1,
                "_id" : ObjectId("554a13d4b692088a38f01f3a"),
                "status" : "live"
            }, 
            {
                "title" : "task 1 sub 4",
                "content" : "aaa",
                "order" : 8,
                "_id" : ObjectId("554a13d4b692088a38f01f4a"),
                "status" : "live"
            }, 
            {
                "title" : "task 1 sub 3 ",
                "content" : "aaa",
                "order" : 2,
                "_id" : ObjectId("5550d0a61662211332d9a973"),
                "status" : "live"
            }
        ]
    }, 
    {
        "title" : "task 2",
        "description" : "task desc 2",
        "order" : 1,
        "_id" : ObjectId("5550855f9ee2db4e3958d299"),
        "status" : "live",
        "subTasks" : [ 
            {
                "title" : "task 2 sub 1",
                "content" : "bbb",
                "order" : 1,
                "_id" : ObjectId("55508f459ee2db4e3958d29a"),
                "status" : "live"
            }
        ]
    }, 
    {
        "title" : "task 3",
        "description" : "task 3 desc",
        "order" : 2,
        "_id" : ObjectId("5551b844bb343a620f85f323"),
        "status" : "live",
        "subTasks" : [ 
            {
                "title" : "task 3 sub 2",
                "content" : "cccc",
                "order" : 0,
                "_id" : ObjectId("5551b88abb343a620f85f324"),
                "status" : "live"
            }, 
            {
                "title" : "task 3 sub 4",
                "content" : "cccc",
                "order" : 1,
                "_id" : ObjectId("5551b8f1bb343a620f85f325"),
                "status" : "hidden"
            }, 
            {
                "title" : "task 3 sub 3",
                "content" : "ccc",
                "order" : 2,
                "_id" : ObjectId("5551ba40bb343a620f85f327"),
                "status" : "hidden"
            }, 
            {
                "title" : "task 3 sub 1",
                "content" : "cccc",
                "order" : -1,
                "_id" : ObjectId("5551bcb8c31283c051d30b7c"),
                "status" : "hidden"
            }
        ]
    }
]

}

我正在使用 Mongodb 聚合管道对其中的任务和子任务进行排序。这是我目前所拥有的：

    db.products.aggregate([
    {
        $project: {
            "tasks" : 1
        }
    },
    {
        $match: {
            _id: ObjectId("554a13d4b692088a38f01f3b")
        }
    },
    {
        $unwind: "$tasks"
    },
    {
        $project: {
            "tasks": 1,
            "subTasks": 1
        }
    },
    {
        $unwind: "$tasks.subTasks"
    },
    {
        $sort: {
            "tasks.subTasks.order": 1
        }
    },
    {
        $sort: {
            "tasks.order": 1
        }
    }
])

结果：

{
"result": [
    {
        "_id": ObjectId("554a13d4b692088a38f01f3b"),
        "tasks": {
            "title": "task 2",
            "description": "task desc 2",
            "order": 1,
            "_id": ObjectId("5550855f9ee2db4e3958d299"),
            "status": "live",
            "subTasks": {
                "title": "task 2 sub 1",
                "content": "bbb",
                "order": 1,
                "_id": ObjectId("55508f459ee2db4e3958d29a"),
                "status": "live"
            }
        }
    },
    {
        "_id": ObjectId("554a13d4b692088a38f01f3b"),
        "tasks": {
            "title": "task 3",
            "description": "task 3 desc",
            "order": 2,
            "_id": ObjectId("5551b844bb343a620f85f323"),
            "status": "live",
            "subTasks": {
                "title": "task 3 sub 1",
                "content": "cccc",
                "order": -1,
                "_id": ObjectId("5551bcb8c31283c051d30b7c"),
                "status": "hidden"
            }
        }
    },
    {
        "_id": ObjectId("554a13d4b692088a38f01f3b"),
        "tasks": {
            "title": "task 3",
            "description": "task 3 desc",
            "order": 2,
            "_id": ObjectId("5551b844bb343a620f85f323"),
            "status": "live",
            "subTasks": {
                "title": "task 3 sub 2",
                "content": "cccc",
                "order": 0,
                "_id": ObjectId("5551b88abb343a620f85f324"),
                "status": "live"
            }
        }
    },
    {
        "_id": ObjectId("554a13d4b692088a38f01f3b"),
        "tasks": {
            "title": "task 3",
            "description": "task 3 desc",
            "order": 2,
            "_id": ObjectId("5551b844bb343a620f85f323"),
            "status": "live",
            "subTasks": {
                "title": "task 3 sub 4",
                "content": "cccc",
                "order": 1,
                "_id": ObjectId("5551b8f1bb343a620f85f325"),
                "status": "hidden"
            }
        }
    },
    {
        "_id": ObjectId("554a13d4b692088a38f01f3b"),
        "tasks": {
            "title": "task 3",
            "description": "task 3 desc",
            "order": 2,
            "_id": ObjectId("5551b844bb343a620f85f323"),
            "status": "live",
            "subTasks": {
                "title": "task 3 sub 3",
                "content": "ccc",
                "order": 2,
                "_id": ObjectId("5551ba40bb343a620f85f327"),
                "status": "hidden"
            }
        }
    },
    {
        "_id": ObjectId("554a13d4b692088a38f01f3b"),
        "tasks": {
            "title": "task 1",
            "description": "task 1 desc",
            "order": 10,
            "_id": ObjectId("554a13d4b692088a38f01f3a"),
            "status": "live",
            "subTasks": {
                "title": "task 1 sub 1",
                "content": "aaa",
                "order": -2,
                "_id": ObjectId("554a13d4b692088a38f01f5a"),
                "status": "live"
            }
        }
    },
    {
        "_id": ObjectId("554a13d4b692088a38f01f3b"),
        "tasks": {
            "title": "task 1",
            "description": "task 1 desc",
            "order": 10,
            "_id": ObjectId("554a13d4b692088a38f01f3a"),
            "status": "live",
            "subTasks": {
                "title": "task 1 sub 2",
                "content": "aaa",
                "order": 1,
                "_id": ObjectId("554a13d4b692088a38f01f3a"),
                "status": "live"
            }
        }
    },
    {
        "_id": ObjectId("554a13d4b692088a38f01f3b"),
        "tasks": {
            "title": "task 1",
            "description": "task 1 desc",
            "order": 10,
            "_id": ObjectId("554a13d4b692088a38f01f3a"),
            "status": "live",
            "subTasks": {
                "title": "task 1 sub 3 ",
                "content": "aaa",
                "order": 2,
                "_id": ObjectId("5550d0a61662211332d9a973"),
                "status": "live"
            }
        }
    },
    {
        "_id": ObjectId("554a13d4b692088a38f01f3b"),
        "tasks": {
            "title": "task 1",
            "description": "task 1 desc",
            "order": 10,
            "_id": ObjectId("554a13d4b692088a38f01f3a"),
            "status": "live",
            "subTasks": {
                "title": "task 1 sub 4",
                "content": "aaa",
                "order": 8,
                "_id": ObjectId("554a13d4b692088a38f01f4a"),
                "status": "live"
            }
        }
    }
],
"ok": 1

}

预期结果：

{
"_id": ObjectId("554a13d4b692088a38f01f3b"),
"title": "product title",
"order": 3,
"description": "Description here ",
"status": "live",
"tasks": [
    {
        "title": "task 2",
        "description": "task desc 2",
        "order": 1,
        "_id": ObjectId("5550855f9ee2db4e3958d299"),
        "status": "live",
        "subTasks": [
            {
                "title": "task 2 sub 1",
                "content": "bbb",
                "order": 1,
                "_id": ObjectId("55508f459ee2db4e3958d29a"),
                "status": "live"
            }
        ]
    },
    {
        "title": "task 3",
        "description": "task 3 desc",
        "order": 2,
        "_id": ObjectId("5551b844bb343a620f85f323"),
        "status": "live",
        "subTasks": [
            {
                "title": "task 3 sub 1",
                "content": "cccc",
                "order": -1,
                "_id": ObjectId("5551bcb8c31283c051d30b7c"),
                "status": "hidden"
            },
            {
                "title": "task 3 sub 2",
                "content": "cccc",
                "order": 0,
                "_id": ObjectId("5551b88abb343a620f85f324"),
                "status": "live"
            },
            {
                "title": "task 3 sub 3",
                "content": "ccc",
                "order": 2,
                "_id": ObjectId("5551ba40bb343a620f85f327"),
                "status": "hidden"
            }{
                "title": "task 3 sub 4",
                "content": "cccc",
                "order": 1,
                "_id": ObjectId("5551b8f1bb343a620f85f325"),
                "status": "hidden"
            }
        ]
    }{
        "title": "task 1",
        "description": "task 1 desc",
        "order": 10,
        "_id": ObjectId("554a13d4b692088a38f01f3a"),
        "status": "live",
        "subTasks": [
            {
                "title": "task 1 sub 1",
                "content": "aaa",
                "order": -2,
                "_id": ObjectId("554a13d4b692088a38f01f5a"),
                "status": "live"
            },
            {
                "title": "task 1 sub 2",
                "content": "aaa",
                "order": 1,
                "_id": ObjectId("554a13d4b692088a38f01f3a"),
                "status": "live"
            },
            {
                "title": "task 1 sub 3 ",
                "content": "aaa",
                "order": 2,
                "_id": ObjectId("5550d0a61662211332d9a973"),
                "status": "live"
            },
            {
                "title": "task 1 sub 4",
                "content": "aaa",
                "order": 8,
                "_id": ObjectId("554a13d4b692088a38f01f4a"),
                "status": "live"
            }
        ]
    }
]

我真的很接近，所有的订购似乎都在工作。我只需要一些帮助才能将子任务放回父母体内。非常感谢任何帮助。

谢谢

【问题讨论】：

聚合的预期/期望输出是什么？
您好，为问题添加了预期结果。谢谢

标签： mongodb mongoose aggregation-framework

【解决方案1】：

您从聚合管道一开始就犯了一个错误

$project: {
            "tasks" : 1
        }

您会因此丢失所有数据。所以首先你需要保留它：

$project: {
             tasks: 1,
             doc: {
                     title: "$title", 
                     order: "$order", 
                     description: "$description", 
                     status: "$status"
                  }
          }

然后像您在问题中所做的那样执行您的$unwinds：

{$unwind: "$tasks"}, {$unwind: "$tasks.subTasks"}

然后进行排序。您需要使用复合键进行排序，否则tasks.subTasks.order 的排序将不会在您按tasks.order 排序时立即成立。所以：

{$sort: {"tasks.order": -1, "tasks.subTasks.order": 1}}

然后是困难的部分。你需要$group返回结果，第一步是$push返回subTasks，但首先，你需要再次保留任务属性：

$project: {
             doc: 1, 
             task_id: "$tasks._id", 
             tasks_doc: {
                           title: "$tasks.title", 
                           description: "$tasks.description", 
                           order: "$tasks.order", 
                           status: "$tasks.status"
                        }, 
             subTasks: "$tasks.subTasks"
          }

...收集subTasks:

$group: {
           _id: {
                   _id: "$_id", 
                   task_id: "$task_id", 
                   doc: "$doc", 
                   task_doc: "$tasks_doc"
                }, 
           subTasks: {
                        $push: "$subTasks"
                     }
        }

tasks 也是如此。请注意，在$grouping 期间，您还需要投射回task_doc 属性：

$group: {
           _id: {
                   _id: "$_id._id", 
                   doc: "$_id.doc"
                }, 
           tasks: {
                     $push: {
                               _id: "$_id.task_id", 
                               title: "$_id.task_doc.title", 
                               description: "$_id.task_doc.description",
                               order: "$_id.task_doc.order", 
                               status: "$_id.task_doc.status" 
                               subTasks: "$subTasks"
                            }
                  }
        }

然后投影回根doc属性：

$project: {
             _id: "$_id._id", 
             title: "$_id.doc.title", 
             description: "$_id.doc.description", 
             order: "$_id.doc.order", 
             status: "$_id.doc.status", 
             tasks: 1
          }

基本上就是这样。这是完整的原始聚合管道，因此您可以测试并查看是否获得所需的结果：

[
 {$match: {_id: ObjectId("554a13d4b692088a38f01f3b")}}, 
 {$project: {tasks: 1, doc: {title: "$title", order: "$order", description: "$description", status: "$status"}}}, 
 {$unwind: "$tasks"}, 
 {$unwind: "$tasks.subTasks"}, 
 {$sort: {"tasks.order": -1, "tasks.subTasks.order": 1}}, 
 {$project: {doc: 1, task_id: "$tasks._id", tasks_doc: {title: "$tasks.title", description: "$tasks.description", order: "$tasks.order", status: "$tasks.status"}, subTasks: "$tasks.subTasks"}}, 
 {$group: {_id: {_id: "$_id", task_id: "$task_id", doc: "$doc", task_doc: "$tasks_doc"}, subTasks: {$push: "$subTasks"}}}, 
 {$group: {_id: {_id: "$_id._id", doc: "$_id.doc"}, tasks: {$push: {_id: "$_id.task_id", title: "$_id.task_doc.title", description: "$_id.task_doc.description", order: "$_id.task_doc.order", status: "$_id.task_doc.status", subTasks: "$subTasks"}}}}, 
 {$project: {_id: "$_id._id", title: "$_id.doc.title", description: "$_id.doc.description", order: "$_id.doc.order", status: "$_id.doc.status", tasks: 1}}
]

更新

如果数组字段为空或不存在（为null），则对该字段will return empty result 进行$unwind 操作。这种情况的解决方案是最初将null/empty 字段设置为某个zero 值，例如"<empty-array>"。请注意，您必须在 $unwind 之前为每个数组执行此 $projection。

看看this answer 了解如何使用$ifNull 运算符。另请查看$size 运算符here。

处理完这部分后，你需要$group返回结果，这可以使用$cond operator来实现，检查"<empty-array>"的值

【讨论】：

虽然这是一个很好的答案 (+1)，但我想补充几点。第一个错误不在管道中，而是在建模中。过度使用嵌入（在 OPs 数据模型中得到很好的例证）很可能导致产品达到 BSON 文档的 16MB 大小限制，除了频繁且昂贵的文档迁移非常可能（如果不确定的话）这一事实。
感谢您指出。在保证初始文档小于 16MB 的情况下，我假设聚合管道的所有中间文档也符合大小限制，因为我们不添加任何数据，而且我们拆分了初始文档。所以我认为大小限制没有问题，还是我遗漏了什么？
或者你的意思是初始数据模型？
我指的是初始数据模型；）
我有一个 products 集合和一个 tasks 集合，并使用 materialized paths 来引用父母，并使用一个字段来引用相应的产品。