使用 ansible 规范化平面数据结构

Question

我目前正在尝试对从 terraform 输出接收到的平面数据结构进行规范化，以便后续操作可以更好地处理信息。在我工作的地方，我倾向于将 Ansible 用于这些类型的任务，但如果它可以大大简化任务，我愿意接受使用 Python 的解决方案。下面是一个示例输入（已清理）以及该输入的所需输出。

输入：

{
  "ebs_block_device.0000000000.device_name": "/dev/xvdc",
  "ebs_block_device.0000000000.volume_id": "vol-00000000000000000",
  "ebs_block_device.0000000000.volume_size": "120",
  "ebs_block_device.0000000000.volume_type": "standard",
  "ebs_block_device.1111111111.device_name": "/dev/xvde",
  "ebs_block_device.1111111111.volume_id": "vol-11111111111111111",
  "ebs_block_device.1111111111.volume_size": "80",
  "ebs_block_device.1111111111.volume_type": "standard",
  "ebs_block_device.2222222222.device_name": "/dev/xvdf",
  "ebs_block_device.2222222222.volume_id": "vol-22222222222222222",
  "ebs_block_device.2222222222.volume_size": "50",
  "ebs_block_device.2222222222.volume_type": "standard",
}

期望的输出：

{
  "devices": [
    {
      "device_name": "/dev/xvdc",
      "volume_id": "vol-00000000000000000",
      "volume_size": "120",
      "volume_type": "standard"
    },
    {
      "device_name": "/dev/xvde",
      "volume_id": "vol-11111111111111111",
      "volume_size": "80",
      "volume_type": "standard"
    },
    {
      "device_name": "/dev/xvdf",
      "volume_id": "vol-22222222222222222",
      "volume_size": "50",
      "volume_type": "standard",
    }   
  ]  
}

迄今为止的尝试

使用以下 ansible 我能够接近所需的输出，但我似乎无法找到正确的过滤器、json_query 过滤器或其他类似的技巧来更改键。

任务：

- name: "Get input from file"
  set_fact:
    device_data: "{{ lookup('file', file_path) | from_json | dict2items }}"

- name: "Extract list of volume id numbers created by terraform"
  vars:
    ebs_regex: "ebs_block_device\\.(\\d*)\\.device_name"
  set_fact:
    volume_id_list: 
      "{{ device_data 
      | selectattr('key', 'match', ebs_regex) 
      | map(attribute='key')
      | map('regex_replace', ebs_regex, '\\1')
      | list }}"

# using '| to_json | from_json |' is a known workaround for a common string typing issue in json_query
# For more information see https://github.com/ansible/ansible/issues/27299
- name: "Organize volumes into list"
  vars:
    query_text: "[?contains(key, '{{ item }}')]"
    single_volume: "{{ device_data | to_json | from_json | json_query(query_text) }}"
  set_fact:
    volume_data: "{{ volume_data | default([]) + [single_volume | items2dict] }}"
  loop:
    "{{ volume_id_list }}"

- debug: var=volume_data

输出：

ok: [localhost] => {
    "volume_data": [
        {
            "ebs_block_device.2659407853.device_name": "/dev/xvdf"
            "ebs_block_device.2659407853.volume_id": "vol-00000000000000000",
            "ebs_block_device.2659407853.volume_size": "50",
            "ebs_block_device.2659407853.volume_type": "standard"
        },
        {
            "ebs_block_device.2630216116.device_name": "/dev/xvde"
            "ebs_block_device.2630216116.volume_id": "vol-11111111111111111",
            "ebs_block_device.2630216116.volume_size": "80",
            "ebs_block_device.2630216116.volume_type": "standard"
        },
        {
            "ebs_block_device.2554893574.device_name": "/dev/xvdc"
            "ebs_block_device.2554893574.volume_id": "vol-22222222222222222",
            "ebs_block_device.2554893574.volume_size": "120",
            "ebs_block_device.2554893574.volume_type": "standard"
        }
    ]
}

问题

如何在构建所需数据结构之前或之后更改结果中键的值？如果这不可能，是否有另一种方法可以将输入数据标准化为所需的格式？

Answer 1

下面的戏

tasks:
  - set_fact:
      devices1: "{{ devices1|default([]) + [ item ] }}"
    with_together:
      - "{{ input|dict2items|sort|selectattr('key', 'search', 'device_name')|list }}"
      - "{{ input|dict2items|sort|selectattr('key', 'search', 'volume_id')|list }}"
      - "{{ input|dict2items|sort|selectattr('key', 'search', 'volume_size')|list }}"
      - "{{ input|dict2items|sort|selectattr('key', 'search', 'volume_type')|list }}"
  - include_tasks: loop-devices.yml
    loop: "{{ devices1 }}"
  - debug:
      var: devices2

包含文件 loop-devices.yml

- set_fact:
    dev: {}
- set_fact:
    dev: "{{ dev|combine({dev_item.key.split('.').2:dev_item.value}) }}"
  loop: "{{ item }}"
  loop_control:
    loop_var: dev_item
- set_fact:
    devices2: "{{ devices2|default([]) + [ dev ] }}"

给予：

"devices2": [
    {
        "device_name": "/dev/xvdc", 
        "volume_id": "vol-00000000000000000", 
        "volume_size": "120", 
        "volume_type": "standard"
    }, 
    {
        "device_name": "/dev/xvde", 
        "volume_id": "vol-11111111111111111", 
        "volume_size": "80", 
        "volume_type": "standard"
    }, 
    {
        "device_name": "/dev/xvdf", 
        "volume_id": "vol-22222222222222222", 
        "volume_size": "50", 
        "volume_type": "standard"
    }
]