Python JSON 按日期键过滤并写入新的 JSON 文件

Question

我有一个 Python 脚本，它会读取一些 JSON 文件，然后将它们导入 MongoDB。

我希望它只插入具有 Published 键 1 个月或更短时间的记录。

我当前的代码是：-

import json
import logging
import logging.handlers
import os
import pymongo
from pymongo import MongoClient


def import_json(mongo_server,mongo_port, vuln_folder):
    try:
        logging.info('Connecting to MongoDB')
        client = MongoClient(mongo_server, mongo_port)
        db = client['vuln_sets']
        coll = db['vulnerabilities']
        logging.info('Connected to MongoDB')
        basepath = os.path.dirname(__file__)
        filepath = os.path.abspath(os.path.join(basepath, ".."))
        archive_filepath = filepath + vuln_folder
        filedir = os.chdir(archive_filepath)
        file_count = 0
        for item in os.listdir(filedir):
            if item.endswith('.json'):
                file_name = os.path.abspath(item)
                with open(item, 'r') as currentfile:
                    vuln_counter = 0
                    duplicate_count = 0
                    logging.info('Currently processing ' + item)
                    file_count +=1
                    json_data = currentfile.read()
                    vuln_content = json.loads(json_data)
                    for vuln in vuln_content:
                        try:
                            del vuln['_type']
                            new_vuln = {key: vuln[key] for key in vuln if key != '_source'}
                            new_vuln.update(vuln['_source'])
                            coll.insert(new_vuln, continue_on_error=True)
                            vuln_counter +=1
                        except pymongo.errors.DuplicateKeyError:
                            duplicate_count +=1

                logging.info('Added ' + str(vuln_counter) + ' vulnerabilities for ' + item)
                logging.info('Found ' + str(duplicate_count) + ' duplicate records!')
                os.remove(file_name)
        logging.info('Processed ' + str(file_count) + ' files')
    except Exception as e:
        logging.exception(e)

我想我可以做一个 IF 语句（伪代码！）：

filter_vuln = if vuln.published = datetime.now -1:
              coll.insert(filter_vuln)

我猜它会丢弃任何与该模式不匹配的记录？

JSON 看起来像这样：

[
  {
    "_index": "bulletins",
    "_type": "bulletin",
    "_id": "OPENWRT-SA-000001",
    "_score": null,
      "lastseen": "2016-09-26T15:45:23",
      "references": 
      "affectedPackage": [
        {
          "OS": "OpenWrt",
          "OSVersion": "15.05",
          "packageVersion": "9.9.8-P3-1",
          "packageFilename": "UNKNOWN",
          "arch": "all",
          "packageName": "bind",
          "operator": "lt"
        }
      ],
      "edition": 1,
      "description": "Some Description",
      "reporter": "OpenWrt Project",
      "published": "2016-01-24T13:33:41",
      "modified": "2016-01-24T13:33:41",
  },

为简洁起见，已从上述 JSON 中删除了一些数据，因为实际记录很长，这是较短的记录之一！

Answer 1

我猜当您说上个月内是指过去 30 天时，对于此示例，您需要 timedelta。

从日期时间导入时间增量，日期时间

今天 = datetime.now()

lastmonth = today - timedelta(days=30)

测试 = ['2017-11-21','2017-10-20']

测试日期：

if date >= str(lastmonth):
    print(date)
else:
    pass

结果是：2017-11-21

这只是一个关于如何按日期过滤的示例