【问题标题】:What type of tree traversal does the ast module use?ast 模块使用什么类型的树遍历?
【发布时间】:2014-05-04 00:02:57
【问题描述】:

ast 使用什么类型的树遍历(特别是ast.NodeVisitor())?当我创建一个堆栈并将遍历的每个节点推入堆栈时,结果似乎是“广度优先”树遍历。这意味着顺序取决于树中的级别。

例如。树的样子

Module
  Assign
    Name
      Store
    Call
      Attribute
        Str
        Load

堆栈看起来像

[Module,Assign,Name,Call,Store,Attribute,Str,Load]

例如。代码

stack = []
class a(ast.NodeTransformer):
    def visit_Num(self,node):
        stack.append(node)
        ...
        return node

    ...                      #this is all the other visit_*() functions

    def visit_Str(self,node):
        stack.append(node)
        ...
        return node

if __name__ == "__main__":
    with open('some_file.py','r') as pt:
        tree = ast.parse(pt)
    new_tree = a()
    new_tree_edit = ast.fix_missing_locations(new_tree.visit(tree)) # I have tried with and without calling fix_missing_locations and got the same results.
    print stack

【问题讨论】:

  • 您是指ast.walk() 函数,还是ast.NodeVisitor()
  • @Martijn Pieters - 我的意思是 ast.NodeVisitor(),但你知道 ast.walk() 方法是否使用不同的遍历吗?

标签: python stack abstract-syntax-tree tree-traversal


【解决方案1】:

ast.walk() 函数走树呼吸优先;见ast.py source:

def walk(node):
    """
    Recursively yield all descendant nodes in the tree starting at *node*
    (including *node* itself), in no specified order.  This is useful if you
    only want to modify nodes in place and don't care about the context.
    """
    from collections import deque
    todo = deque([node])
    while todo:
        node = todo.popleft()
        todo.extend(iter_child_nodes(node))
        yield node

新节点被推入队列,下一个被遍历的节点是队列的最前面。

如果您想要深度优先遍历,请改用ast.NodeVisitor() 的子类;它将使用递归遍历树; NodeVisitor.visit() 调用 NodeVisitor.generic_visit() 除非定义了更特定于节点的访问者方法,并且 NodeVisitor.generic_visit() 再次为子节点调用 NodeVisitor.visit()

class NodeVisitor(object):
    """
    A node visitor base class that walks the abstract syntax tree and calls a
    visitor function for every node found.  This function may return a value
    which is forwarded by the `visit` method.

    This class is meant to be subclassed, with the subclass adding visitor
    methods.

    Per default the visitor functions for the nodes are ``'visit_'`` +
    class name of the node.  So a `TryFinally` node visit function would
    be `visit_TryFinally`.  This behavior can be changed by overriding
    the `visit` method.  If no visitor function exists for a node
    (return value `None`) the `generic_visit` visitor is used instead.

    Don't use the `NodeVisitor` if you want to apply changes to nodes during
    traversing.  For this a special visitor exists (`NodeTransformer`) that
    allows modifications.
    """

    def visit(self, node):
        """Visit a node."""
        method = 'visit_' + node.__class__.__name__
        visitor = getattr(self, method, self.generic_visit)
        return visitor(node)

    def generic_visit(self, node):
        """Called if no explicit visitor function exists for a node."""
        for field, value in iter_fields(node):
            if isinstance(value, list):
                for item in value:
                    if isinstance(item, AST):
                        self.visit(item)
            elif isinstance(value, AST):
                self.visit(value)

如果您将 NodeVisitor 子类化(或者它的派生版本 NodeTransformer),请记住在您的特定 visit_* 方法中也调用 super(YourClass, self).generic_visit(node) 以继续遍历树。

【讨论】:

  • 我上面得到的结果是使用了一个扩展 ast.NodeTransformer 的类。然后在 visit_*() 方法中将每个新节点推入堆栈。所以这让我觉得 ast.NodeVisitor 使用广度优先搜索。抱歉这么含糊,但我错过了什么吗?我将发布一些示例代码。
  • @baallezx:您的NodeTransformer 子类似乎根本没有访问子节点;每个visit_* 方法都必须调用super(a, self).generic_visit(node) 才能访问子节点。
  • @MartijnPieters 如何遍历 AST 并获取所有路径(从根到其终端节点)?通用视图可以做到这一点吗?
  • @tumbleweed:不知道你为什么要这么做? NodeVisitor 不访问路径,它首先访问节点深度。您可以将堆栈保留为实例属性,在调用self.visit(item) 之前将item 添加到堆栈顶部,并在self.visit(item) 返回时再次将其从堆栈中弹出,然后该堆栈就是完整的“路径”从根到当前访问的节点。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2014-08-16
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2022-01-11
相关资源
最近更新 更多