在python中解析json答案

【问题标题】：Parsing json in python在python中解析json
【发布时间】：2013-12-09 10:41:41
【问题描述】：

我无法解析以下 json 文件。我正在尝试使用 logstash/python 解析它。

{
  "took" : 153,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 946,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "incoming_bytes",
      "_type" : "logs",
      "_id" : "lZSq4mBRSVSxO0kyTwh3fQ",
      "_score" : 1.0, "_source" : {"user_id":"86c8c25d81a448c49e3d3924ea5ceddf","name":"network.incoming.bytes","resource_id":"instance-00000001-c8be5ca1-116b-45b3-accb-1b40050abc90-tapaf1e421f-c8","timestamp":"2013-11-02T07:32:36Z","resource_metadata":{"name":"tapaf1e421f-c8","parameters":{},"fref":null,"instance_id":"c8be5ca1-116b-45b3-accb-1b40050abc90","instance_type":"5c014e54-ee16-43a8-a763-54e243bd8969","mac":"fa:16:3e:67:39:29"},"volume":557462,"source":"openstack","project_id":"9ac587404bdd4fcdafe41c0b10f9f8ae","type":"cumulative","id":"f1eb19aa-4390-11e3-8bac-000c2973cfb1","unit":"B","@timestamp":"2013-11-02T07:32:38.276Z","@version":"1","host":"127.0.0.1","tags":["_grokparsefailure"],"priority":13,"severity":5,"facility":1,"facility_label":"user-level","severity_label":"Notice","@type":"%{appdeliveryserver}"}
    }, {
      "_index" : "incoming_bytes",
      "_type" : "logs",
      "_id" : "073URWt5Sc-krLACxQnI3g",
      "_score" : 1.0, "_source" : {"user_id":"86c8c25d81a448c49e3d3924ea5ceddf","name":"network.incoming.bytes","resource_id":"instance-00000001-c8be5ca1-116b-45b3-accb-1b40050abc90-tapaf1e421f-c8","timestamp":"2013-11-02T07:32:38Z","resource_metadata":{"name":"tapaf1e421f-c8","parameters":{},"fref":null,"instance_id":"c8be5ca1-116b-45b3-accb-1b40050abc90","instance_type":"5c014e54-ee16-43a8-a763-54e243bd8969","mac":"fa:16:3e:67:39:29"},"volume":562559,"source":"openstack","project_id":"9ac587404bdd4fcdafe41c0b10f9f8ae","type":"cumulative","id":"f31e38d4-4390-11e3-8bac-000c2973cfb1","unit":"B","@timestamp":"2013-11-02T07:32:39.001Z","@version":"1","host":"127.0.0.1","tags":["_grokparsefailure"],"priority":13,"severity":5,"facility":1,"facility_label":"user-level","severity_label":"Notice","@type":"%{appdeliveryserver}"}
    }]
  }
}

我对logstash使用了以下配置，但是配置没有按预期工作：解析JSON文档中的各个字段并输出到STDOUT。

input {
 stdin{}
  file {

        path => ["/home/****/Downloads/throughput"]
        codec => "json"
    }
}
filter{
    json{

        source => "message"
        target => "throughput"

    }
}
output {
  stdout {codec => rubydebug }

}

对于 python，我正在尝试访问单个卷和源（IP 地址）字段。

我尝试了以下代码，目标是为每条记录映射各个字段，我想知道如何继续遍历和提取列表中的各个元素。

import json
from pprint import pprint

json_data=open('throughput')

data = json.load(json_data)

pprint(data["hits"])

json_data.close()

谢谢

【问题讨论】：

您的具体问题是什么？ IE。什么不工作，什么方法？
Logstash：Logstash 不向 stdout 提供任何输出，我检查了 json 验证器，格式有效。 Python：我想知道如何继续遍历和提取各个字段。

标签： python json logstash

【解决方案1】：

解析出来的json是dictionary，你可以使用itemgetter向下钻取。

例如音量

>>> for hits in data['hits']['hits']:
...     print hits['_source']['volume']
...
557462
562559

或者可以使用map获取列表：

>>> from operator import itemgetter
>>> map(itemgetter('volume'), map(itemgetter('_source'), data['hits']['hits']))
[557462, 562559]

【讨论】：

谢谢，昨天自己搞定了。当它起作用时，我也有关于 Logstash 失败的想法。有没有cmets？