将单个 Json 文件数据保存到同一 SQL DB 中的 2 个不同表中

Question

我需要将 json 文件数据保存到 mysql db。目前 mysql db mydb 带有 table1 和 table2。需要将数据json文件的一部分保存到table1，一部分保存到table2。每个 table1 和 table2 都已创建。

以下是json数据文件示例

{
  "response": {
    "dev_log": {
      "data": [
        {
          "id": "1",
          "timestamp": "2020-01-16 10:11:12",
          "email": "johnd@gmail.com"
        },
        {
          "id": "2",
          "timestamp": "2020-02-27 15:33:34",
          "email": "zack@gmail.com"
        },
        {
          "id": "3",
          "timestamp": "2020-02-27 15:34:07",
          "email": "edy@yahoo.com"
        }
      ],
      "total_dev_log": "1423"
    },
    "client_log": {
      "data": [
        {
          "customer_city": "LONDON",
          "customer_login": "AAAAAAAAAAAAAA",
          "customer_state": "MC",
          "details": "aaaaaaaaaaa-bbbbbbbbbbbbbbb-cccccccccccccc ",
          "log_number": "1",
          "dept": "Sales",
          "staff_id": "S123",
          "staff_name": "EricY",
          "timestamp": "2020-02-27 15:57:24"
        },
        {
          "customer_city": "SINGAPORE",
          "customer_login": "BBBBBBBBBBBBB",
          "customer_state": "XX",
          "details": "ddddddddd-eeeeeeeeeeee-ffffffffffff ",
          "log_number": "1",
          "dept": "Eng",
          "staff_id": "S456",
          "staff_name": "YongG",
          "timestamp": "2020-02-27 15:57:24"
        }
      ],
      "total_hero_query": "13"
    },
    "response_time": "0.723494",
    "transaction_id": "909122",
    "transaction_status": "OK",
    "transaction_time": "Fri Feb 28 15:27:51 2020"
  }
}

这里...在 json 中，我们有 'dev_log' 和 'client_log'。 因此..dev_log 的所有值都应该保存到 mydb 数据库的 table1 和 client_log 到 table2 上。

下面草拟的代码

import pymysql
import os
import json
#import ast

#Read Json string file
with open('datfile.json', 'r') as f:
   datDict = json.load(f)

#connect to MySQL
con = pymysql.connect(host = 'localhost',user = 'root',passwd = 'root',db = 'mydb')
cursor = con.cursor()

#Parse data to SQL insert
#for i, item in enumerate(datDict):
    #id = ("id", None)
    #timestamp = ("timestamp", None)
    #email = ("email", None)

    #cursor.execute("INSERT INTO mytable (id,   timestamp,   email) VALUES (%s,    %s,     %s)", (id,  timestamp,   email))
cursor.executemany("""INSERT INTO table1 VALUES(id, timestamp, email)""", datDict['response']['dev_log']['data'])
con.commit()
con.close()

我不确定如何将文件保存到 sql 以及更多到 2 个不同的表。至于现在..我可以运行代码没有任何错误，但返回空值如下；

mysql> SELECT * FROM table1;
+------+-----------+-------+
| id   | timestamp | email |
+------+-----------+-------+
| NULL | NULL      | NULL  |
| NULL | NULL      | NULL  |
| NULL | NULL      | NULL  |
+------+-----------+-------+
3 rows in set (0.00 sec)

mydb 数据库和表 table1 已创建。

非常感谢您的帮助，并建议我如何进一步执行。

谢谢大家

Answer 1

Json 只是一种文本表示，因此您应该首先将其加载到 Python 对象（字典和列表）中。完成后，data['response']['dev_log']['data'] 是一个不错的字典列表，data['response']['client_log']['data'] 也是如此。

所以您应该能够使用以下查询：

data = json.load(jsonfile)
...
curs = con.cursor()    # con is a connection to your database...
curs.executemany("""INSERT INTO dev VALUES(:id, :timestamp, :email)""",
    data['response']['dev_log']['data'])