灵活地将新数据附加到 yaml 文件

Question

我有不同的 yaml 文件，它们可能有不同的嵌套结构

file1.yaml:

test3:
  service1: 
      name1: |
        "somedata"
      name2: |
          "somedata"

file2.yaml:

test1: 
  app1: 
     app2:|
       "somedata"
  app7:
     key2: | 
       "testapp"

如您所见，yaml 文件的结构可能有所不同。

问题是，我可以以某种方式灵活地管理将一些数据附加到这些文件的特定块吗？

例如在 file1 中，我想在 name1 和 name 2 键或 service1 级别上写入键值：

test3:
  service1: 
      name1: |
        "somedata"
      name2: |
          "somedata"
      my-appended-key:| 
              "my appended value"
  my_second_appended_key: | 
          "my second appended valye"

等等。

所以这个想法是能够指定我想在 yaml 中的哪个嵌套块下附加数据。

我有不同的 yaml 文件，它们可能有不同的嵌套结构

file1.yaml:

test3:
  service1: 
      name1: |
        "somedata"
      name2: |
          "somedata"

file2.yaml:

test1: 
  app1: 
     app2:|
       "somedata"
  app7:
     key2: | 
       "testapp"

如您所见，yaml 文件的结构可能有所不同。

问题是，我可以以某种方式灵活地管理将一些数据附加到这些文件的特定块吗？

例如在 file1 中，我想在 name1 和 name 2 键或 service1 级别上写入键值：

test3:
  service1: 
      name1: |
        "somedata"
      name2: |
          "somedata"
      my-appended-key:| 
              "my appended value"
  my_second_appended_key: | 
          "my second appended valye"

等等。

现在我针对 yaml 文件结构的特定情况进行处理。 这是我的代码的一部分：

import gnupg
import re
import argparse

def NewPillarFile():
    with open(args.sensitive) as sensitive_data:
        with open(args.encrypted, "w") as encrypted_result:
            encrypted_result.write('#!yaml|gpg\n\nsecrets:\n    '+args.service+':\n')
            for line in sensitive_data:
                encrypted_value = gpg.encrypt(re.sub(r'^( +?|[A-Za-z0-9]|[A-Za]|[0-9])+( +)?'+args.separator+'( +)?','',line,1), recipients=args.resident, always_trust=True)
                if not encrypted_value.ok:
                    print(encrypted_value.status, '\n', encrypted_value.stderr)
                    break
                line = re.sub(r'^( +)?','',line)
                encrypted_result.write('        '+re.sub(r'( +)?'+args.separator+'.*',': |',line))
                encrypted_result.write(re.sub(re.compile(r'^', re.MULTILINE), '            ', encrypted_value.data.decode())+'\n')

def ExistingPillarFile():
    with open(args.sensitive) as sensitive_data:
        with open(args.encrypted, "a") as encrypted_result:
            encrypted_result.write('    '+args.service+':\n')
            for line in sensitive_data:
                encrypted_value = gpg.encrypt(
                    re.sub(r'^( +?|[A-Za-z0-9]|[A-Za]|[0-9])+( +)?' + args.separator + '( +)?', '', line, 1),
                    recipients=args.resident, always_trust=True)
                if not encrypted_value.ok:
                    print(encrypted_value.status, '\n', encrypted_value.stderr)
                    break
                line = re.sub(r'^( +)?', '', line)
                encrypted_result.write('        ' + re.sub(r'( +)?' + args.separator + '.*', ': |', line))
                encrypted_result.write(re.sub(re.compile(r'^', re.MULTILINE), '            ', encrypted_value.data.decode())+'\n')

所以想法是能够指定在 yaml 中的哪个嵌套块下我想附加数据以使脚本更灵活。

Answer 1

您可以使用 PyYAML 的低级事件接口。假设您有一个输入 YAML 文件并希望将修改写入输出 YAML 文件，您可以编写一个函数，通过 PyYAML 生成的事件流并在指定位置插入请求的附加值：

import yaml
from yaml.events import *

class AppendableEvents:
  def __init__(self, path, events):
    self.path = path
    self.events = events

  def correct_position(self, levels):
    if len(self.path) != len(levels):
      return False
    for index, expected in enumerate(self.path):
      if expected != levels[index].cur_id:
        return False
    return True

class Level:
  def __init__(self, mode):
    self.mode = mode
    self.cur_id = -1 if mode == "item" else ""

def append_to_yaml(yamlFile, targetFile, items):
  events = []
  levels = []
  with open(yamlFile, 'r') as handle:
    for event in yaml.parse(handle):
      if isinstance(event, StreamStartEvent) or \
         isinstance(event, StreamEndEvent) or \
         isinstance(event, DocumentStartEvent) or \
         isinstance(event, DocumentEndEvent):
        pass
      elif isinstance(event, CollectionStartEvent):
        if len(levels) > 0:
          if levels[-1].mode == "key":
            # we can only handle scalar keys
            raise ValueError("encountered complex key!")
          else:
            if levels[-1].mode == "value":
              levels[-1].mode = "key"
        if isinstance(event, MappingStartEvent):
          levels.append(Level("key"))
        else: # SequenceStartEvent
          levels.append(Level("item"))
      elif isinstance(event, ScalarEvent):
        if len(levels) > 0:
          if levels[-1].mode == "item":
            levels[-1].cur_id += 1
          elif levels[-1].mode == "key":
            levels[-1].cur_id = event.value
            levels[-1].mode = "value"
          else: # mode == "value"
            levels[-1].mode = "key"
      elif isinstance(event, CollectionEndEvent):
        # here we check whether we want to append anything
        levels.pop()
        for item in items:
          if item.correct_position(levels):
            for additional_event in item.events:
              events.append(additional_event)
      events.append(event)
  with open(targetFile, mode="w") as handle:
    yaml.emit(events, handle)

要使用它，您必须提供要附加为 YAML 事件列表的其他内容，并将所需位置指定为键列表（或序列索引）：

def key(name):
  return ScalarEvent(None, None, (True, True), name)

def literal_value(content):
  return ScalarEvent(None, None, (False, True), content, style="|")

append_to_yaml("file1.yaml", "file1_modified.yaml", [
  AppendableEvents(["test3", "service1"], [
    key("my-appended-key"),
    literal_value("\"my appended value\"\n")]),
  AppendableEvents(["test3"], [
    key("my_second_appended_key"),
    literal_value("\"my second appended value\"\n")])])

此代码正确地将您的file1.yaml 转换为给定的修改文件。通常，这还允许您附加复杂的（序列或映射）节点。下面是一个基本示例：

def seq(*scalarValues):
  return [SequenceStartEvent(None, None, True)] + \
    [ScalarEvent(None, None, (True, False), v) for v in scalarValues] + \
    [SequenceEndEvent()]

def map(*scalarValues):
  return [MappingStartEvent(None, None, True)] + \
    [ScalarEvent(None, None, (True, False), v) for v in scalarValues] + \
    [MappingEndEvent()]

append_to_yaml("file1.yaml", "file1_modified.yaml", [
  AppendableEvents(["test3", "service1"], [
    key("my-appended-key")] + seq("one", "two", "three")),
  AppendableEvents(["test3"], [
    key("my_second_appended_key")] + map("foo", "bar"))])