【问题标题】:Solving Overembedding MongoDB with Slow Down performance解决过度嵌入 MongoDB 并降低性能
【发布时间】:2017-01-18 00:45:59
【问题描述】:

目前我正在一个项目中工作,该项目存在严重的过度嵌入问题,因为在其中只有一个集合中有 3 个对象(数组),其中几乎包含应用程序 70% 的业务模型(我们收到了这个来自其他开发团队的项目,是一个完整的挑战)。另一个问题是,该应用程序使用地理定位进行实时跟踪,并持续使用此集合。

我的建议是,我完全确定数据库服务器中的过载问题和应用程序在几个小时内的速度变慢是由于这个原因造成的过度嵌入。

我们认为解决方案是创建一个新的数据库模式(不知道 MongoDB 是无模式但不是限制),尝试在低引用的树集合中规范这三个对象(模拟关系模型等外键),但是,例如,您建议使用旧(当前)数据库设计和制作数据仓库,只进行读取查询并仅迁移用户数据或将所有数据库迁移到新模型(可能非常复杂......或不是?)......

附加信息: 公交车集合统计

{
    "ns" : "pruebas.buses",
    "count" : 1343,
    "size" : 38393616,
    "avgObjSize" : 28587,
    "numExtents" : 7,
    "storageSize" : 58277888,
    "lastExtentSize" : 20643840.0,
    "paddingFactor" : 1.0,
    "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
    "userFlags" : 1,
    "capped" : false,
    "nindexes" : 1,
    "totalIndexSize" : 65408,
    "indexSizes" : {
        "_id_" : 65408
    },
    "ok" : 1.0
}

这是来自这个名为 Buses 的集合中的文档示例:

{
    "_id" : "BAOB-02",
    "school" : "BAOBAB",
    "licensePlate" : "UFS 118",
    "color" : "BLANCO",
    "model" : 2002,
    "username" : "baobab02",
    "students" : [ 
        {
            "firstNames" : "MATTHIAS ",
            "lastNames" : "GARCIA VELANDIA",
            "_id" : "1002",
            "classroom" : "",
            "blood" : "",
            "telephone" : null,
            "cellphone" : null,
            "guardians" : [ 
                {
                    "firstNames" : "GUSTAVO ",
                    "lastNames" : "GARCIA GARAVITO",
                    "_id" : ObjectId("553515248a854eba40c1d2fc")
                }, 
                {
                    "firstNames" : "CLAUDIA ",
                    "lastNames" : "VELANDIA ",
                    "_id" : ObjectId("553515248a854eba40c1d2fb")
                }
            ],
            "parents" : [ 
                {
                    "firstNames" : "GUSTAVO ",
                    "lastNames" : "GARCIA GARAVITO",
                    "telephone" : null,
                    "cellphone" : 3103247894.0,
                    "email" : "gggzipa@gmail.com",
                    "_id" : ObjectId("553515248a854eba40c1d2fe")
                }, 
                {
                    "firstNames" : "CLAUDIA ",
                    "lastNames" : "VELANDIA ",
                    "telephone" : null,
                    "cellphone" : 3102487056.0,
                    "email" : "ar.claudiavelandia@gmail.com",
                    "_id" : ObjectId("553515248a854eba40c1d2fd")
                }
            ],
            "addressInfo" : {
                "pm" : {
                    "address" : "KM 2 TABIO - CAJICA",
                    "apartment" : "",
                    "neighborhood" : "VIA TABIO",
                    "monday" : true,
                    "tuesday" : true,
                    "wednesday" : true,
                    "thursday" : true,
                    "friday" : true,
                    "saturday" : false,
                    "coords" : [ 
                        4.9242399390697, 
                        -74.0441983938217
                    ],
                    "stopOrder" : 1
                },
                "am" : {
                    "address" : "NA",
                    "apartment" : "",
                    "neighborhood" : "",
                    "monday" : false,
                    "tuesday" : false,
                    "wednesday" : false,
                    "thursday" : false,
                    "friday" : false,
                    "saturday" : false,
                    "coords" : []
                }
            },
            "code" : "1002"
        }, 
        {
            "firstNames" : "JUAN PABLO",
            "lastNames" : "ROMERO GUZMAN",
            "_id" : "1003",
            "classroom" : "",
            "blood" : "",
            "telephone" : null,
            "cellphone" : null,
            "guardians" : [ 
                {
                    "firstNames" : "NELSON ANDRES",
                    "lastNames" : "ROMERO ",
                    "_id" : ObjectId("5535158b8a854eba40c1d300")
                }, 
                {
                    "firstNames" : "ANA MARIA",
                    "lastNames" : "GUZMAN MORENO",
                    "_id" : ObjectId("5535158b8a854eba40c1d2ff")
                }
            ],
            "parents" : [ 
                {
                    "firstNames" : "NELSON ANDRES",
                    "lastNames" : "ROMERO ",
                    "telephone" : null,
                    "cellphone" : 3192997309.0,
                    "email" : "nelsonandresromerojimenez@hotmail.com",
                    "_id" : ObjectId("5535158b8a854eba40c1d302")
                }, 
                {
                    "firstNames" : "ANA MARIA",
                    "lastNames" : "GUZMAN MORENO",
                    "telephone" : null,
                    "cellphone" : 3143095644.0,
                    "email" : "ananita28@hotmail.com",
                    "_id" : ObjectId("5535158b8a854eba40c1d301")
                }
            ],
            "addressInfo" : {
                "pm" : {
                    "address" : "CRR 7 2 46",
                    "apartment" : "APT. 404 INT. 8",
                    "neighborhood" : "CAPELLANIA",
                    "monday" : true,
                    "tuesday" : true,
                    "wednesday" : true,
                    "thursday" : true,
                    "friday" : true,
                    "saturday" : false,
                    "coords" : [ 
                        4.91861203215498, 
                        -74.0340435504913
                    ],
                    "stopOrder" : 2
                },
                "am" : {
                    "address" : "NA",
                    "apartment" : "",
                    "neighborhood" : "",
                    "monday" : false,
                    "tuesday" : false,
                    "wednesday" : false,
                    "thursday" : false,
                    "friday" : false,
                    "saturday" : false,
                    "coords" : []
                }
            },
            "code" : "1003"
        }
    ],
    "auxiliary" : {
        "firstNames" : "LEIDY VIVIANA",
        "lastNames" : "MORANTES BARON",
        "telephone" : null,
        "cellphone" : 3203178186.0,
        "email" : "vivis_120490@hotmail.com"
    },
    "driver" : {
        "firstNames" : "VICTOR JULIO",
        "lastNames" : "MORANTES MORANTES",
        "telephone" : null,
        "cellphone" : 3118955381.0
    },
    "__v" : 13
}

此集合包含学生 +- 18,每个学生通常有 2 个父母。目前存在 1300 个文档。实时地理定位跟踪的数据分配在另一个集合中,但项目使用另一个服务器进行 REDIS 缓存(我知道缓存所有数据库不是一个好习惯,但我们计划将此缓存分段仅用于跟踪服务)

以及所有数据库的统计数据>

{
    "db" : "pruebas",
    "collections" : 20,
    "objects" : 5785288,
    "avgObjSize" : 285.557788652873,
    "dataSize" : 1652034048.0,
    "storageSize" : 2388484096.0,
    "numExtents" : 112,
    "indexes" : 18,
    "indexSize" : 176544368.0,
    "fileSize" : 4226809856.0,
    "nsSizeMB" : 16,
    "extentFreeList" : {
        "num" : 0,
        "totalSize" : 0
    },
    "dataFileVersion" : {
        "major" : 4,
        "minor" : 22
    },
    "ok" : 1.0
}

PD/ 一个月前,我们可能会为 MongoDB 应用优化技术,例如 Sharding 或使用负载均衡器和 Mongos 进行复制……但无论如何,我们知道如果数据库设计错误,解决问题的最佳形式是做一个新模型。 谢谢,如果有人花时间阅读所有这些奇怪的案例.. 如果提出建设性的 cmets 和建议,请提前感谢

【问题讨论】:

    标签: mongodb redis data-warehouse node-redis real-time-data


    【解决方案1】:

    在上面的描述中,没有提到根本原因,并且看起来基于假设,brodriguezs 正在走向架构更改

    修改架构前的一些提示。

    您可以在这里看到一些重要提示 - https://docs.mongodb.com/manual/administration/analyzing-mongodb-performance/

    【讨论】:

    • 感谢您的建议。我将审查链接。目前monggoDB版本是3.0.7,Storage engine mmapv1,并没有replication,这个可以推荐吗?因为我们有另一台带有 redis 的服务器连接到缓存数据库,但是对于某些查询来说太慢了。我将检查关于索引文档的第一点,但是我认为嵌入式集合可能不太有用。但我会考虑您对新数据库的建议。谢谢..!
    猜你喜欢
    • 2019-11-15
    • 2015-03-04
    • 2018-12-28
    • 1970-01-01
    • 2023-03-22
    • 1970-01-01
    • 2020-06-02
    • 1970-01-01
    • 2019-05-21
    相关资源
    最近更新 更多