【问题标题】:Use insert_one is not woring when data include datetime.datetime当数据包含 datetime.datetime 时,使用 insert_one 不起作用
【发布时间】:2020-05-04 05:03:01
【问题描述】:

我使用Python scrapy(1.8.0) 抓取一些数据并将其保存到mongodbpymongo

当我使用Python2 和更早的pymongo 版本时,它可以工作。

但是当我尝试使用Python3pymongo 3.10.1 时,我发现插入文档不起作用。

我发现问题来自datetime

像这样:

'releasedTime': [datetime.datetime(2020, 1, 17, 14, 30),datetime.datetime(2020, 1, 16, 18, 10),datetime.datetime(2020, 1, 17, 22, 0)]

日期数据来自如下:

stripSingleTime = singleTime.strip() # 14:30
myTime = datetime.datetime.strptime(stripSingleTime, '%H:%M').time() # 14:30:00
myDateTime = datetime.datetime.combine(datetime.date.today(), myTime) # 2020-01-17 14:30:00
# x is my for loop argument
myDateTimeArray[x].append(myDateTime)  # [[datetime.datetime(2020, 1, 17, 14, 30)], []]

这里是关于insert_one

import pymongo
from pymongo import MongoClient
from scrapy.utils.project import get_project_settings
settings = get_project_settings()
import datetime

class MongoDBPipeline(object):

    global theaters
    theaters = []

    def __init__(self):
        connection = MongoClient(
            settings['MONGODB_SERVER'],
            settings['MONGODB_PORT'])
        self.db = connection[settings['MONGODB_DB']]
        self.collection = self.db[settings['MONGODB_COLLECTION']]
    def open_spider(self, spider):
        print ('Pipelines => open_spider =>', spider)

    def process_item(self, item, spider):
        global theaters

        self.collection = self.db[type(item).__name__.replace('_Item','')]

        if  item['theater'] not in theaters:
            theaters.append(item['theater'])
            self.collection.remove({'theater': item['theater']})

        # self.collection.insert_one(mydict) is working
        mydict = { 'name': 'RUNOOB', 'alexa': '10001', 'url': 'https://www.runoob.com' }

        test = {
        'geometry': {'coordinates': [120.196866, 22.99322], 'type': 'Point'}, 
        'phone': '06-2205151',
        'theater': 'TodayTainan',
        'movie': [{
        'enName': 'The Lion King',
        'goodMinePoint': 0.75,
        'imdbScore': '8.5',
        'photoHref': 'https://movies.yahoo.com.tw/x/r/w420/i/o/production/movies/June2019/M07RYsvcBWpi3xJtjCQF-2714x3878.jpg',
        'releasedTime': [datetime.datetime(2020, 1, 17, 14, 30)],
        'rottenScore': '93%',
        'videoId': ['MQuUkET0lQg', 'he2rj_8XwsE']
        }]
        }

        self.collection.insert_one(test) # insert the dummy object test is not working

        self.collection.create_index([("geometry", pymongo.GEOSPHERE)])

        return item

为什么self.collection.insert_one(test) 不起作用?

我猜pymongomongodb 之间有问题。他们不处理我的datetime

任何帮助将不胜感激。谢谢。

【问题讨论】:

  • 定义“不工作”。它会出错吗?文件没有插入但没有错误?插入的文档包含意外数据?
  • 如果我在self.collection.insert_one(test) 之后设置 print('test')。它不会被打印出来。如果我设置 try except 与self.collection.insert_one(test),代码将运行捕捉,但 print(err) 什么都没有。

标签: python-3.x mongodb scrapy pymongo


【解决方案1】:

改变

    self.collection.insert_one(test) # insert the dummy object test is not working

    import traceback
    try:
        result = self.collection.insert_one(test) # insert the dummy object test is not working
        print (result.inserted_id)
    except Exception:
        traceback.print_exc()

并报告输出。

【讨论】:

  • 谢谢肚皮!我试试你的代码,看到输出显示KeyError: 'AntispamItem does not support field: _id',我找到了解决方案。谢谢。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2021-04-07
  • 1970-01-01
  • 1970-01-01
  • 2015-07-12
  • 2018-02-18
  • 2018-10-12
  • 1970-01-01
相关资源
最近更新 更多