【发布时间】：2020-05-12 19:04:36
【问题描述】：
过去一周我已经尽我所能尝试通过 python 3 将 JSON 文件加载到 Postgres 表中。它是一个嵌套的 JSON，虽然我能够完成一些基本示例，但我无法做到正确编码。我的 ado.json 文件是这样的。最终我只想把这个东西弄平。
我不断收到此错误：
xecute_sql() error: syntax error at or near "{"
LINE 2: VALUES ('System.LinkTypes.Hierarchy', {'linkType': 'System.L...
我对 python 很陌生，只是在动态中学习。看来我调用列表的方式只是读取 2 列（rel 和属性）。我已经阅读了几十个博客，但不幸的是我无法正确地阅读。
[
{
"rel":"System.LinkTypes.Hierarchy","attributes":{"linkType":"System.LinkTypes.Hierarchy-Forward","sourceId":13,"targetId":23,"isActive":true,"changedDate":"2019-01-18T18:45:53.013Z","changedBy":{"id":"3209f8e3-95a2-6448-a146-13e374bd03bc","displayName":"Stacey Clark","uniqueName":"Clark_Stacey@gaaf.com","descriptor":"aad.MzIwOWY4ZTMtOTVhMi03NDQ4LWExNDYtMTNlMzc0YmQwM2Jj"},"comment":null,"changedOperation":"create","sourceProjectId":"7dc32e0c-84d4-46a4-aec6-1f0a22b60ef8","targetProjectId":"7dc32e0c-84d4-46a4-aec6-1f0a22b60ef8"}
},
{
"rel":"System.LinkTypes.Hierarchy","attributes":{"linkType":"System.LinkTypes.Hierarchy-Forward","sourceId":9,"targetId":24,"isActive":true,"changedDate":"2019-01-18T18:46:08.64Z","changedBy":{"id":"3209f8e3-95a2-6448-a146-13e374bd03bc","displayName":"Stacey Clark","uniqueName":"Clark_Stacey@gaaf.com","descriptor":"aad.MzIwOWY4ZTMtOTVhMi03NDQ4LWExNDYtMTNlMzc0YmQwM2Jj"},"comment":null,"changedOperation":"create","sourceProjectId":"7dc32e0c-84d4-46a4-aec6-1f0a22b60ef8","targetProjectId":"7dc32e0c-84d4-46a4-aec6-1f0a22b60ef8"}
},
{
"rel":"System.LinkTypes.Hierarchy","attributes":{"linkType":"System.LinkTypes.Hierarchy-Forward","sourceId":9,"targetId":25,"isActive":true,"changedDate":"2019-01-18T18:46:26.64Z","changedBy":{"id":"3209f8e3-95a2-6448-a146-13e374bd03bc","displayName":"Stacey Clark","uniqueName":"Clark_Stacey@gaaf.com","descriptor":"aad.MzIwOWY4ZTMtOTVhMi03NDQ4LWExNDYtMTNlMzc0YmQwM2Jj"},"comment":null,"changedOperation":"create","sourceProjectId":"7dc32e0c-84d4-46a4-aec6-1f0a22b60ef8","targetProjectId":"7dc32e0c-84d4-46a4-aec6-1f0a22b60ef8"}
}
]
我使用的 Python 脚本如下。
import json, sys # Import Python's built-in JSON Library
import pandas as pd
import flatten_json
from pandas.io.json import json_normalize
from psycopg2 import connect, Error # import the psycopg2 database adapter for PostgreSQL
# use Python's open() function to load the JSON data
with open('ado.json', encoding='utf-8') as json_data:
record_list = json.loads(json_data.read())
print(record_list)
if type(record_list) == list:
first_record = record_list[0]
# I am unable to get the correct column names. only rel and attributes show up
columns = list(first_record.keys())
print ("\ncolumn names:", columns)
table_name = "json_data_ado"
sql_string = 'INSERT INTO {} '.format( table_name )
sql_string += "(" + ', '.join(columns) + ")\nVALUES "
for i, record_dict in enumerate(record_list):
values = []
for col_names, val in record_dict.items():
# Postgres strings must be enclosed with single quotes
if type(val) == str:
val = val.replace("'", "''")
val = "'" + val + "'"
values += [ str(val) ]
# join the list of values and enclose record in parenthesis
sql_string += "(" + ', '.join(values) + "),\n"
# remove the last comma and end statement with a semicolon
sql_string = sql_string[:-2] + ";"
##insert json data into postgres sql -- Simply output to screen
print ("\nSQL statement:")
print (sql_string)
# Connect to postgres
try:
# declare a new PostgreSQL connection object
conn = connect(
dbname = "postgres",
user = "postgres",
host = "test.us-east-1.rds.amazonaws.com",
password = "postgres",
# attempt to connect for 3 seconds then raise exception
connect_timeout = 10
)
cur = conn.cursor()
print ("\ncreated cursor object:", cur)
except (Exception, Error) as err:
print ("\npsycopg2 connect error:", err)
conn = None
cur = None
if cur != None:
try:
cur.execute( sql_string )
conn.commit()
print ('\nfinished INSERT INTO execution')
except (Exception, Error) as error:
print("\nexecute_sql() error:", error)
conn.rollback()
cur.close()
conn.close()
【问题讨论】：
-
Use bind parameters，而不是字符串连接，将值放入 SQL 查询中。然后所有的转换代码都消失了。
标签： json python-3.x postgresql nested