使用数据集从 Hive 中的字符串中提取 json 字段答案

【问题标题】：Extracting json field from string in Hive using dataset使用数据集从 Hive 中的字符串中提取 json 字段
【发布时间】：2019-05-30 18:25:42
【问题描述】：

我正在尝试一个非常基本的配置单元查询。我正在尝试从数据集中提取 json 字段，但我总是得到 p>

\N

对于 json 字段，但是 some_string 没问题

这是我的查询：

WITH dataset AS (
SELECT
CAST(
   '{ "traceId": "abc", "additionalData": "{\"Star Rating\":\"3\"}",  "locale": "en_US", "content": { "contentType": "PB", "content": "T S", "bP": { "mD": { "S R": "3" }, "cType": "T_S", "sType": "unknown-s", "bTimestamp": 0, "title": "T S" } }
    }' AS STRING) AS some_string
)
SELECT some_string, get_json_object(dataset.some_string, '$.traceId') FROM dataset

问题：如何在此处获取 json 字段？

【问题讨论】：

有效：demo.gethue.com/hue/editor?editor=342256
对不起，我想我把它简化了。 json 中的附加数据给出了问题。虽然它是一个有效的 json。仍然失败。我已更新查询以将 addtionalData 添加到查询中的 json

标签： json hive hiveql qubole

【解决方案1】：

问题在于反斜杠。单个反斜杠被视为 " 的转义字符并被 Hive 删除：

hive> select '\"';
OK
"
Time taken: 0.069 seconds, Fetched: 1 row(s)

当你有两个反斜杠时，Hive 会删除一个：

hive> select '\\"';
OK
\"
Time taken: 0.061 seconds, Fetched: 1 row(s)

使用两个反斜杠可以正常工作：

WITH dataset AS (
  SELECT
  CAST(
     '{ "traceId": "abc", "additionalData": "{\\"Star Rating\\":\\"3\\"}",  "locale": "en_US", "content": { "contentType": "PB", "content": "T S", "bP": { "mD": { "S R": "3" }, "cType": "T_S", "sType": "unknown-s", "bTimestamp": 0, "title": "T S" } }
       }' AS STRING) AS some_string
   )
   SELECT some_string,  get_json_object(dataset.some_string, '$.traceId') FROM dataset;
OK
{ "traceId": "abc", "additionalData": "{\"Star Rating\":\"3\"}",  "locale": "en_US", "content": { "contentType": "PB", "content": "T S", "bP": { "mD": { "S R": "3" }, "cType": "T_S", "sType": "unknown-s", "bTimestamp": 0, "title": "T S" } }
    }   abc
Time taken: 0.788 seconds, Fetched: 1 row(s)

您还可以在附加数据中轻松删除 { 之前和 } 之后的双引号：

WITH dataset AS (
SELECT
regexp_replace(regexp_replace(
   '{ "traceId": "abc", "additionalData": "{\"Star Rating\":\"3\"}",  "locale": "en_US", "content": { "contentType": "PB", "content": "T S", "bP": { "mD": { "S R": "3" }, "cType": "T_S", "sType": "unknown-s", "bTimestamp": 0, "title": "T S" } }
    }' ,'\\"\\{','\\{') ,'\\}\\"','\\}' )AS some_string
)
SELECT some_string,  get_json_object(dataset.some_string, '$.traceId') FROM dataset;

OK
{ "traceId": "abc", "additionalData": {"Star Rating":"3"},  "locale": "en_US", "content": { "contentType": "PB", "content": "T S", "bP": { "mD": { "S R": "3" }, "cType": "T_S", "sType": "unknown-s", "bTimestamp": 0, "title": "T S" } }
    }   abc
Time taken: 7.035 seconds, Fetched: 1 row(s)

【讨论】：