mongoDB：从 pymongo 查询时失败分组答案

【问题标题】：mongoDB: group by failing while querying from pymongomongoDB：从 pymongo 查询时失败分组
【发布时间】：2011-11-17 00:13:41
【问题描述】：

这是我正在做的事情：

>>> import pymongo
>>> con = pymongo.Connection('localhost',12345)
>>> db = con['staging']
>>> coll = db['contract']
>>> result = coll.group(['asset_id'], None, {'list': []}, 'function(obj, prev) {prev.list.push(obj)}') 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.macosx-10.3-fat/egg/pymongo/collection.py", line 908, in group
  File "build/bdist.macosx-10.3-fat/egg/pymongo/database.py", line 340, in command
  File "build/bdist.macosx-10.3-fat/egg/pymongo/helpers.py", line 126, in _check_command_response
pymongo.errors.OperationFailure: command SON([('group', {'$reduce': Code('function(obj, prev) {prev.list.push(obj)}', {}), 'ns': u'contract', 'cond': None, 'key': {'asset_id': 1}, 'initial': {'list': []}})]) failed: exception: BufBuilder grow() > 64MB

我在 mongod 日志上看到的内容如下

Wed Nov 16 16:05:55 [conn209] Assertion: 13548:BufBuilder grow() > 64MB                                                                           0x10008de9b 0x100008d89 0x100151e72 0x100152712 0x100151954 0x100152712 0x100151954 0x100152712 0x100152e7b 0x100152f0c 0x10013b1d9 0x1003706bf 0x10037204c 0x10034c4d6 0x10034d877 0x100180cc4 0x100184649 0x1002b9e89 0x1002c3f18 0x100433888                                                      0   mongod                              0x000000010008de9b _ZN5mongo11msgassertedEiPKc + 315                                                      1   mongod                              0x0000000100008d89 _ZN5mongo10BufBuilder15grow_reallocateEv + 73
 2   mongod                              0x0000000100151e72 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 2962
 3   mongod                              0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
 4   mongod                              0x0000000100151954 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 1652
 5   mongod                              0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
 6   mongod                              0x0000000100151954 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 1652
 7   mongod                              0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
 8   mongod                              0x0000000100152e7b _ZN5mongo9Convertor8toObjectEl + 139
 9   mongod                              0x0000000100152f0c _ZN5mongo7SMScope9getObjectEPKc + 92
 10  mongod                              0x000000010013b1d9 _ZN5mongo11PooledScope9getObjectEPKc + 25
 11  mongod                              0x00000001003706bf _ZN5mongo12GroupCommand5groupESsRKSsRKNS_7BSONObjES3_SsSsPKcS3_SsRSsRNS_14BSONObjBuilderE + 3551
 12  mongod                              0x000000010037204c _ZN5mongo12GroupCommand3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb + 3676
 13  mongod                              0x000000010034c4d6 _ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 1350
 14  mongod                              0x000000010034d877 _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi + 2151
 15  mongod                              0x0000000100180cc4 _ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi + 52
 16  mongod                              0x0000000100184649 _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 10585
 17  mongod                              0x00000001002b9e89 _ZN5mongo13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 569
 18  mongod                              0x00000001002c3f18 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE + 1528
 19  mongod                              0x0000000100433888 _ZN5mongo10connThreadEPNS_13MessagingPortE + 616
Wed Nov 16 16:05:55 [conn209] query staging.$cmd ntoreturn:1 command: { group: { $reduce: CodeWScope( function(obj, prev) {prev.list.push(obj)}, {}), ns: "contract", cond: null, key: { asset_id: 1 }, initial: { list: {} } } } reslen:111 1006ms

我对pymongo和Mongodb都很陌生，不知道如何解决这个问题，请帮忙

谢谢

【问题讨论】：

标签： mongodb pymongo

【解决方案1】：

堆栈跟踪的相关部分是：

exception: BufBuilder grow() > 64MB

基本上，Mongo 不允许您拥有任何大于 64MB 的文档。请参阅this question 了解更多详情（从那时起，大小限制已提高到 64MB。）

我不确定您要对该查询做什么。看起来您想要获取每个asset_id 的对象列表。但是，您的集合将超过容量，因为您永远不会区分组中的对象。尝试将您的initial 设置为{'asset_id': '', 'objects': []} 和reduce 设置为function(obj, prev) {prev.asset_id = obj.asset_id; prev.objects.push(obj)，尽管有更有效的方法来执行此查询。

或者，如果您要获取与 ID 匹配的所有文档，请尝试：

coll.find({'asset_id': whatevs})

如果您尝试获取对象的数量，请尝试以下操作：

coll.group(
    ['asset_id'], None, {'asset_id': '', 'count': 0}, 
    'function(obj, prev) {prev.asset_id = obj.asset_id; prev.count += obj.count}'
)

【讨论】：