【问题标题】:How to Generate Python Bindings for Thrift in Bazel如何在 Bazel 中为 Thrift 生成 Python 绑定
【发布时间】:2017-11-17 00:44:30
【问题描述】:

我正在尝试使用 Bazel 为 Thrift 服务定义生成 Python 绑定。据我所知,没有现有的.bzl 可以这样做,所以我在这里有点靠我自己。我过去写过.bzl 规则,但我在这种情况下遇到的情况不同。

一般问题是在构建开始之前我不知道来自thrift 命令的输出文件的名称,这意味着我无法生成具有正确设置srcs 属性的py_library 规则因为我没有文件的名称。我尝试遵循示例,通过生成.zip 文件提前知道输出文件,但py_library 规则只允许.py 文件作为srcs,所以这不起作用。

我唯一能想到的就是使用repository_rule 来生成代码和BUILD 文件,但我想要完成的工作似乎并不难,应该得到支持。

【问题讨论】:

  • 嗨,布莱恩,我也遇到了类似的情况。我问了类似的问题here你能分享一下你最后的方法是什么吗?

标签: python code-generation thrift bazel


【解决方案1】:

以前有人尝试过。

在这里讨论: https://groups.google.com/forum/#!topic/bazel-dev/g3DVmhVhNZs

这里的代码: https://github.com/wt/bazel_thrift

我会从那里开始。

编辑: 我从那里开始。我没有达到我希望的那样。 Bazel 正在扩展以支持由一个输入生成多个输出,但它目前还不能很容易地实现这一点,每:

groups.google.com/forum/#!topic/bazel-discuss/3WQhHm194yU

无论如何,我确实为 C++ 节俭绑定尝试了一些东西,它们有同样的问题。 Java 示例通过使用源 jar 作为构建源解决了这个问题,这对我们不起作用。为了让它工作,我传入了我关心的源文件列表,这些源文件将由 thrift 生成器创建。然后我将这些文件报告为将在 impl 中生成的输出。这似乎行得通。有点讨厌,因为您必须在构建之前知道要查找的文件,但它确实有效。也可以让一个小程序读取 thift 文件并确定它将生成的输出文件。那会更好,但我没有时间。另外,当前的方法很好,因为它明确定义了您要寻找 thrift 生成的文件,这使得像我这样的新手更容易理解 BUILD 文件。

先传递一些代码,也许我会清理它并作为补丁提交(也许不会):

###########
# CPP gen 
###########

# Create Generated cpp source files from thrift idl files.
# 
def _gen_thrift_cc_src_impl(ctx):
    out = ctx.outputs.outs

    if not out:
        # empty set
        # nothing to do, no inputs to build
        return DefaultInfo(files=depset(out))

    # Used dir(out[0]) to see what
    # we had available in the object. 
    # dirname attribute tells us the directory
    # we should be putting stuff in, works nicely.
    # ctx.genfile_dir is not the output directory
    # when called as an external repository
    target_genfiles_root = out[0].dirname 
    thrift_includes_root = "/".join(
        [ target_genfiles_root, "thrift_includes"])
    gen_cpp_dir = "/".join([target_genfiles_root,"." ])

    commands = []

    commands.append(_mkdir_command_string(target_genfiles_root))
    commands.append(_mkdir_command_string(thrift_includes_root))

    thrift_lib_archive_files = ctx.attr.thrift_library._transitive_archive_files
    for f in thrift_lib_archive_files:
        commands.append(
            _tar_extract_command_string(f.path, thrift_includes_root))

    commands.append(_mkdir_command_string(gen_cpp_dir))
    thrift_lib_srcs = ctx.attr.thrift_library.srcs
    for src in thrift_lib_srcs:
        commands.append(_thrift_cc_compile_command_string(
            thrift_includes_root, gen_cpp_dir, src))


    inputs = (
        list(thrift_lib_archive_files) + thrift_lib_srcs )

    ctx.action(
        inputs = inputs,
        outputs = out,
        progress_message = "Generating CPP sources from thift archive %s" % target_genfiles_root,
        command = " && ".join(commands),
    )
    return DefaultInfo(files=depset(out))


thrift_cc_gen_src=  rule(
    _gen_thrift_cc_src_impl,
        attrs = {
            "thrift_library": attr.label(
                mandatory=True, providers=['srcs', '_transitive_archive_files']),
            "outs" : attr.output_list(mandatory=True, non_empty=True),
        },output_to_genfiles = True,
    )

# wraps cc_library to generate a library from one or more .thrift files
#   provided as a thrift_library bundle.
# 
# Generates all src and hdr files needed, but you must specify the expected
# files. This is a bug in bazel: https://groups.google.com/forum/#!topic/bazel-discuss/3WQhHm194yU
# 
# Instead of src and hdrs, requires: cpp_srcs and cpp_hdrs. These are required.
#
# Takes:
#           name: The library name, like cc_library
#  
# thrift_library: The library of source .thrift files from which our
#                 code will be built from.
# 
#       cpp_srcs: The expected source that will be generated and built. Passed to
#                 cc_library as src.
#
#       cpp_hdrs: The expected header files that will be generated. Passed to 
#                 cc_library as hdrs.
#
# Rest of options are documented in native.cc_library 
#

def thrift_cc_library(name, thrift_library, 
                      cpp_srcs=[],cpp_hdrs=[],
                      build_skeletons=False,
                      deps=[], alwayslink=0, copts=[], 
                      defines=[], include_prefix=None, 
                      includes=[], linkopts=[], 
                      linkstatic=0, nocopts=None,
                      strip_include_prefix=None,
                      textual_hdrs=[],
                      visibility=None):
    # from our thrift_library tarball source bundle, 
    # create a generated cpp source directory.
    outs = []
    for src in cpp_srcs:
        outs.append("//:"+src)
    for hdr in cpp_hdrs:    
        outs.append("//:"+hdr)
    thrift_cc_gen_src( 
        name = name + 'cc_gen_src',
        thrift_library = thrift_library,
        outs = outs,
    )

    # Then make the library for the given name.
    native.cc_library(
        name = name,
        deps = deps,
        srcs = cpp_srcs,
        hdrs = cpp_hdrs,
        alwayslink = alwayslink,
        copts = copts,
        defines=defines,
        include_prefix=include_prefix,
        includes=includes,
        linkopts=linkopts,
        linkstatic=linkstatic,
        nocopts=nocopts,
        strip_include_prefix=strip_include_prefix,
        textual_hdrs=textual_hdrs,
        visibility=visibility,
    )

【讨论】:

  • 是的,我考虑过明确说明应该生成的文件。这很糟糕,所以尽管我知道它会起作用,但我并没有走那条路。我也想过使用存储库规则,但这似乎令人担忧。最后,我考虑过生成生成文件的.zip 并构建一个存根.py 来解压缩包含源的.zip。我确信它会起作用,但这似乎是一个黑客攻击。
猜你喜欢
  • 1970-01-01
  • 2017-11-21
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多