如何通过字典递归并动态更新值，以便以后可以引用它们？

Question

我正在尝试编写一个配置变量引擎，该引擎接受 YAML 文件（包含 AWS 配置变量）作为输入并转换为 JSON，以便可以将其上传到 HTTP k/v API（例如 Consul）。我被难住的一个功能是允许开发人员在后续键中“包含”键集（用下划线标识，在最终有效负载中省略）。一个例子如下：

# Region
us-east-1:
  # Any key preceded by an underscore (_) is considered a "tag group" and will not be uploaded to Consul KV unless explicitly included.
  _taggroup1:
    key1: value1
    key2: value2
    key3: value3
  _taggroup2:
    key4: value1
    key5: value2
    key6: value3

  dev:
    _include: us-east-1/_taggroup1
  qa:
    _include:
      - us-east-1/_taggroup1
      - us-east-1/_taggroup2
    key6: baz
  prod:
    _include:
      - us-east-1/_taggroup1
      - us-east-1/_taggroup2

us-west-1:
  _taggroup1:
    key1: value1
    key2: value2
    key3: value3
  _taggroup2:
    key4: value1
    key5: value2
    key6: value3

  dev:
    _include:
      - us-west-1/_taggroup1
  qa:
    _include:
      - us-west-1/_taggroup1
      - us-west-1/_taggroup2
    key2: foo
  prod:
    _include:
      - us-west-1/_taggroup1
      - us-west-1/_taggroup2
    key4: foo
    key5: bar
    key1: undef

  us-west-1a:
    qa:
      _include: us-west-1/qa
    prod:
      _include: us-west-1/prod

  us-west-1b:
    _include: us-west-1/us-west-1a

如您所见，我正在尝试构建一个配置文件，允许开发人员对变量进行分组，然后根据需要包含/覆盖它们。

到目前为止，我为此实验编写的代码本质上是您的标准递归函数，并添加了特定于该应用程序的附加功能：

# parse_input is a separate function that converts a YAML stream into
# an OrderedDict
original_dict = parse_input(stream1)

def print_dict(input_dict):

    new_dict = collections.OrderedDict()

    for key, value in input_dict.iteritems():
        if key.startswith('_'):
            if key == '_include':
                if isinstance(value, list):
                    for item in value:
                        x = dpath.util.get(original_dict, item)
                        for k, v in x.iteritems():
                            new_dict[k] = v
                else:
                    x = dpath.util.get(original_dict, value)
                    for k, v in x.iteritems():
                        new_dict[k] = v
            else:
                continue
            continue
        elif isinstance(value, dict):
            new_dict[key] = print_dict(value)
        else:
            new_dict[key] = value
    return new_dict

到目前为止，我已经实现的输出是这样的：

{
    "us-east-1": {
        "dev": {
            "key1": "value1", 
            "key2": "value2", 
            "key3": "value3"
        }, 
        "qa": {
            "key1": "value1", 
            "key2": "value2", 
            "key3": "value3", 
            "key4": "value1", 
            "key5": "value2", 
            "key6": "baz"
        }, 
        "prod": {
            "key1": "value1", 
            "key2": "value2", 
            "key3": "value3", 
            "key4": "value1", 
            "key5": "value2", 
            "key6": "value3"
        }
    }, 
    "us-west-1": {
        "dev": {
            "key1": "value1", 
            "key2": "value2", 
            "key3": "value3"
        }, 
        "qa": {
            "key1": "value1", 
            "key2": "foo", 
            "key3": "value3", 
            "key4": "value1", 
            "key5": "value2", 
            "key6": "value3"
        }, 
        "prod": {
            "key1": "undef", 
            "key2": "value2", 
            "key3": "value3", 
            "key4": "foo", 
            "key5": "bar", 
            "key6": "value3"
        }, 
        "us-west-1a": {
            "qa": {
                "_include": [
                    "us-west-1/_taggroup1", 
                    "us-west-1/_taggroup2"
                ], 
                "key2": "foo"
            }, 
            "prod": {
                "_include": [
                    "us-west-1/_taggroup1", 
                    "us-west-1/_taggroup2"
                ], 
                "key4": "foo", 
                "key5": "bar", 
                "key1": "undef"
            }
        }, 
        "us-west-1b": {
            "qa": {
                "_include": "us-west-1/qa"
            }, 
            "prod": {
                "_include": "us-west-1/prod"
            }
        }
    }
}

如您所见，我似乎已经完成了一半。我的问题是，在我最初的实验中，通过在引用包含集时引用函数中的original_dict 变量（使用dpath 返回键），我得到了有利的结果。这很快变成了一个问题，因为函数递归得更深（即在这种情况下是 AZ 特定的变量），因为我不知道如何动态更新原始字典中的键，或者以其他方式跟踪更改，所以函数将注入带有_include 密钥的密钥集，并且无法重新评估它们。

如何消除对原始字典的依赖，而是动态跟踪更改，以便在树的更深处正确评估 _include 键？

Answer 1

我认为这段代码解决了您面临的问题。关键变化是递归到print_dict，结果来自dpath。我还折叠了一些冗余代码。

代码：

import yaml
import collections
import json
import dpath

with open('data.yml', 'rb') as f:
    original_dict = yaml.load(f)

def print_dict(input_dict):

    new_dict = collections.OrderedDict()

    for key, value in input_dict.iteritems():
        if key.startswith('_'):
            if key == '_include':
                if not isinstance(value, list):
                    value = [value]
                for item in value:
                    x = print_dict(dpath.util.get(original_dict, item))
                    for k, v in x.iteritems():
                        new_dict[k] = v
        elif isinstance(value, dict):
            new_dict[key] = print_dict(value)
        else:
            new_dict[key] = value
    return new_dict

print(json.dumps(print_dict(original_dict), indent=2))

输出：

{
  "us-east-1": {
    "qa": {
      "key3": "value3", 
      "key2": "value2", 
      "key1": "value1", 
      "key6": "baz", 
      "key5": "value2", 
      "key4": "value1"
    }, 
    "prod": {
      "key3": "value3", 
      "key2": "value2", 
      "key1": "value1", 
      "key6": "value3", 
      "key5": "value2", 
      "key4": "value1"
    }, 
    "dev": {
      "key3": "value3", 
      "key2": "value2", 
      "key1": "value1"
    }
  }, 
  "us-west-1": {
    "qa": {
      "key2": "value2", 
      "key3": "value3", 
      "key1": "value1", 
      "key6": "value3", 
      "key5": "value2", 
      "key4": "value1"
    }, 
    "us-west-1b": {
      "qa": {
        "key2": "value2", 
        "key3": "value3", 
        "key1": "value1", 
        "key6": "value3", 
        "key5": "value2", 
        "key4": "value1"
      }, 
      "prod": {
        "key1": "value1", 
        "key3": "value3", 
        "key2": "value2", 
        "key6": "value3", 
        "key5": "bar", 
        "key4": "foo"
      }
    }, 
    "prod": {
      "key1": "value1", 
      "key3": "value3", 
      "key2": "value2", 
      "key6": "value3", 
      "key5": "bar", 
      "key4": "foo"
    }, 
    "us-west-1a": {
      "qa": {
        "key2": "value2", 
        "key3": "value3", 
        "key1": "value1", 
        "key6": "value3", 
        "key5": "value2", 
        "key4": "value1"
      }, 
      "prod": {
        "key1": "value1", 
        "key3": "value3", 
        "key2": "value2", 
        "key6": "value3", 
        "key5": "bar", 
        "key4": "foo"
      }
    }, 
    "dev": {
      "key3": "value3", 
      "key2": "value2", 
      "key1": "value1"
    }
  }
}