逐块解析 JSON

Question

我有一个包含客户列表和日期的 JSON 文件。

文件如下所示：

{
"Customers": [
{
  "Customer": "Customer Name Here",
  "Company": "Super Coffee",
  "First Name": "First Name Here",
  "Main Phone": "777-777-7777",
  "Fax": "777-777-7777",
  "Bill to 1": "Billing Address One",
  "Bill to 2": "Billing Address Two",
  "Bill to 3": "Billing Address Three",
  "Ship to 1": "Shipping Address One",
  "Ship to 2": "Shipping Address Two",
  "Ship to 3": "Shipping Address Three",
  "Customer Type": "Dealer/Retail"
},
{
  "Customer": "Customer Name Here",
  "Company": "Turtle Mountain Welding",
  "First Name": "First Name Here",
  "Main Phone": "777-777-7777",
  "Fax": "777-777-7777",
  "Bill to 1": "Billing Address One",
  "Bill to 2": "Billing Address Two",
  "Bill to 3": "Billing Address Three",
  "Ship to 1": "Shipping Address One",
  "Ship to 2": "Shipping Address Two",
  "Ship to 3": "Shipping Address Three",
  "Customer Type": "Dealer/Retail"
},
{
  "Customer": "Customer Name Here",
  "Company": "Mountain Equipment Coop",
  "First Name": "First Name Here",
  "Main Phone": "777-777-7777",
  "Fax": "777-777-7777",
  "Bill to 1": "Billing Address One",
  "Bill to 2": "Billing Address Two",
  "Bill to 3": "Billing Address Three",
  "Ship to 1": "Shipping Address One",
  "Ship to 2": "Shipping Address Two",
  "Ship to 3": "Shipping Address Three",
  "Customer Type": "Dealer/Retail"
},
{
  "Customer": "Customer Name Here",
  "Company": "Best Soup Inc.",
  "First Name": "First Name Here",
  "Main Phone": "777-777-7777",
  "Fax": "777-777-7777",
  "Bill to 1": "Billing Address One",
  "Bill to 2": "Billing Address Two",
  "Bill to 3": "Billing Address Three",
  "Ship to 1": "Shipping Address One",
  "Ship to 2": "Shipping Address Two",
  "Ship to 3": "Shipping Address Three",
  "Customer Type": "Dealer/Retail"
}
]
}

我需要能够逐块而不是逐行从文件中提取数据。

我习惯于逐行解析文件以获取数据，但是对于 JSON，我需要以某种方式逐块读取它（或者更准确地说，逐个对象？）。我需要为每个客户阅读括号内的内容。这样我就可以编写一个脚本来提取我需要的数据，并从中构建一个 CSV 文件。

例如：

i="1"
for file in *.json; do
     customername=$(jsonblock$i:customername);
     customerAddress=$(jsonblock$i:customeraddress);
     etc...
     i=$[i+1]
done

我明白在逐行读取文件时这是如何完成的，但是我怎样才能像读取一行一样读取每个 JSON 块呢？

Answer 1

对于上面的 JSON（由于提供的数据无效而进行了修改），以下脚本将解析并打印每个块的 "Company:" 部分：

#!/usr/bin/env perl

use JSON;   
use IO::All;     
use v5.16;

my $data < io 'Our_Customers.json';
my $customers_list = decode_json($data)->{"Customers"};                

for my $customer (@$customers_list) {
   say $customer->{"Company"} ;
}

输出：

Super Coffee
Turtle Mountain Welding
Mountain Equipment Coop
Best Soup Inc.

脚本使用IO::All 和JSON 来读取和解析(decode_json) 文件。

在这个例子中，JSON 数据被简单地映射到一个 Perl 数据结构（Array of Hashes），它与 JSON 数据完全对应。然后我们可以访问每个数组元素（i.e array 中的每个 hash），然后通过键名访问散列中的数据。 Perl 具有非常灵活的数据处理和访问功能，这使得处理 JSON 数据变得非常愉快。

每个数据块的键来自 JSON 文件的等效部分。如果我们将一个元素从数组中移出，它将是一个哈希，我们访问可以看到元素的keys 和values，如下所示：

say for keys shift $customers_list ;

Customer Type
First Name
Bill to 2
Main Phone
...

使用您在for 循环中看到的$element->{"key"} 语法访问每个键的值。

---

最好在将 JSON 数据发布到 SO - JSON Lint 之前对其进行验证，类似的服务可以提供帮助。

Answer 2

如果您只是想以 CSV 格式打印 JSON 数据，那么您问错了问题。您应该解析整个 JSON 文档并逐项处理 Customers 数组。

使用 Perl 的 JSON 和 Text::CSV 模块，看起来像这样

use strict;
use warnings;

use JSON 'from_json';
use Text::CSV ();

my @columns = (
  'Bill to 1',  'Bill to 2',     'Bill to 3', 'Company',
  'Customer',   'Customer Type', 'Fax',       'First Name',
  'Main Phone', 'Ship to 1',     'Ship to 2', 'Ship to 3',
);

my $out_fh = \*STDOUT;
my $json_file = 'customers.json';

my $data = do {
  open my $fh, '<', $json_file or die qq{Unable to open "$json_file" for input: $!};
  local $/;
  from_json(<$fh>);
};
my $customers = $data->{Customers};

my $csv = Text::CSV->new({ eol => $/ });
$csv->print($out_fh, \@columns);

for my $customer ( @$customers ) {
  $csv->print($out_fh, [ @{$customer}{@columns} ]);
}

输出

"Bill to 1","Bill to 2","Bill to 3",Company,Customer,"Customer Type",Fax,"First Name","Main Phone","Ship to 1","Ship to 2","Ship to 3"
"Billing Address One","Billing Address Two","Billing Address Three","Super Coffee","Customer Name Here",Dealer/Retail,777-777-7777,"First Name Here",777-777-7777,"Shipping Address One","Shipping Address Two","Shipping Address Three"
"Billing Address One","Billing Address Two","Billing Address Three","Turtle Mountain Welding","Customer Name Here",Dealer/Retail,777-777-7777,"First Name Here",777-777-7777,"Shipping Address One","Shipping Address Two","Shipping Address Three"
"Billing Address One","Billing Address Two","Billing Address Three","Mountain Equipment Coop","Customer Name Here",Dealer/Retail,777-777-7777,"First Name Here",777-777-7777,"Shipping Address One","Shipping Address Two","Shipping Address Three"
"Billing Address One","Billing Address Two","Billing Address Three","Best Soup Inc.","Customer Name Here",Dealer/Retail,777-777-7777,"First Name Here",777-777-7777,"Shipping Address One","Shipping Address Two","Shipping Address Three"

Answer 3

使用 perl 和 JSON 库，您可以逐步解析 JSON 列表中的每个项目，但您需要调整 json 以便它实际上不是 json，而是不以逗号分隔的 json 对象列表。

#!/usr/bin/perl
use strict;
use warnings;
use feature qw(say);
use JSON;
my $json = JSON->new;
while (<>) {
    my $obj_or_undef = eval { $json->incr_parse( $_ ); };
    # Wait until its found a whole object
    if (ref $obj_or_undef) {
        say join ",", map {$obj_or_undef->{$_}} sort keys %$obj_or_undef;
    }
}

对于customers.json（不再完全是json）：

{ 
    "some key" : "some value"
} {
    "other key" : "other value"
}

运行：

$ perl demo.pl < customers.json
some value
other value
$ perl demo.pl < customers.json > customer.csv