我可以使用 SCons 显示构建的进度吗？

Question

我有一个使用SCons 组合在一起的项目的简单构建系统。该项目有相当多的源文件，我认为在构建进行时显示某种进度信息会是用户友好的。 SCons 提供了像 CXXCOMSTR 这样的构造变量，我可以覆盖这些变量来控制在每个构建步骤中显示到终端的内容。例如，而不是看到类似的东西：

gcc file.c -o file.o

如果有类似的东西就好了：

[1/10] Compiling file.c: `gcc file.c -o file.o`

[1/10] 指定这是在此构建期间更新的十个目标中的第一个。有什么方法可以访问这些信息，以便我可以生成这样的消息？似乎我需要知道正在更新的目标的总数（这是基于 SCons 所做的依赖扫描）以及枚举每个目标的某种方式。我知道类似的行为与 CMake 和 waf 等其他构建系统相似，但在 SCons 的（扩展的）文档中没有遇到任何内容。

Answer 1

这可以通过Progress() SCons 函数来完成。

SCons man page中有更多例子，搜索“进度”即可。

您可以提供要调用的函数、将定期显示的字符串或将以旋转方式显示的字符串列表。您可以访问当前正在构建的目标，但我认为不可能知道完成百分比。

Answer 2

我想出了一个相当不错的解决方法，可以让您以百分比的形式显示构建进度。基本上下面的代码是这样工作的：

• 如果以前从未构建过目标，它将在每个目标旁边显示[??%]
• 在第一次构建时，它将缓存在.scons_progress 中为该目标处理的总节点数
• 下次运行构建时，它将在启动时从文件中读取并获取上次来自该目标的总节点并使用它来显示百分比（即[50%]）。
• 如果节点数超过缓存总数，它将恢复为[??%] 并更新.scons_progress
• 随着时间的推移，您的 SConstruct “稳定”（即您不会一直进行更改），这些缓存值应该会越来越准确。
• 更新缓存会减慢您的构建速度（每个节点会稍微变慢），因此您可能需要将 interval 设置为 5，以便它仅每 5 个节点更新一次缓存

代码如下：

在custom_utils.py 下site_scons/:

PROGRESS = {
    "env": None,
    "nodes": 0,
    "interval": 1,
    "total": 0,
    "cache": {},
    "cache_key": "_",
    "cache_name": ".scons_progress",
    "cache_update": False
}

def get_progress_total():
    return PROGRESS["cache"].get(PROGRESS["cache_key"], 0)

def update_progress_cache():
    PROGRESS["cache"][PROGRESS["cache_key"]] = PROGRESS["nodes"]
    with open(PROGRESS["cache_name"], "w") as f:
        f.write(json.dumps(PROGRESS["cache"]))

def load_progress_cache():
    try:
        with open(PROGRESS["cache_name"]) as f:
            PROGRESS["cache"] = json.load(f)

            cache_key = "_".join(COMMAND_LINE_TARGETS)
            if cache_key:
                PROGRESS["cache_key"] = cache_key
    except IOError as e:
        pass

def progress_function(node):
    PROGRESS["nodes"] += PROGRESS["interval"]

    #Only update cached environment if it is not None
    if node.env:
        PROGRESS["env"] = node.env
        PROGRESS["env"].Replace(PROGRESS = "[??%]")

    #Update display message
    if PROGRESS["nodes"] <= PROGRESS["total"]:
        PROGRESS["env"].Replace(PROGRESS = "[%d%%]" % (100*PROGRESS["nodes"]/PROGRESS["total"]))
    #If current node is more than total, we need to update the cache
    elif not PROGRESS["cache_update"]:
        PROGRESS["cache_update"] = True
        PROGRESS["env"].Replace(PROGRESS = "[??%]")
    #If cache flag is on, update cache file
    else:
        update_progress_cache()

def progress_settings(env, interval):
    load_progress_cache()
    PROGRESS["interval"] = interval
    PROGRESS["env"]      = env
    PROGRESS["total"]    = get_progress_total()
    Progress(progress_function, interval=PROGRESS["interval"])

在SConstruct:

if GetOption("progress"):
    custom_utils.progress_settings(env, 5)

样本输出：

user@linuxbox:~/project1$ scons -j4 --progress
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
[??%] Compiling (shared): src/ExpressRuntimeException.os
[??%] Compiling (shared): src/Application.os
[??%] Compiling (shared): src/Callback.os
[??%] Compiling (shared): src/Response.os
[??%] Compiling (shared): src/Router.os
[??%] Compiling (shared): src/Route.os
[??%] Compiling (shared): src/Regex.os
[??%] Compiling (shared): src/tools.os
[??%] Linking   (shared): libexpresscpp.so
scons: done building targets.

user@linuxbox:~/project1$ scons -c
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Cleaning targets ...
Removed src/ExpressRuntimeException.os
Removed src/Application.os
Removed src/Callback.os
Removed src/Response.os
Removed src/Router.os
Removed src/Route.os
Removed src/Regex.os
Removed src/tools.os
Removed libexpresscpp.so
scons: done cleaning targets.

user@linuxbox:~/project1$ scons -j4 --progress
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
[14%] Compiling (shared): src/ExpressRuntimeException.os
[51%] Compiling (shared): src/Application.os
[59%] Compiling (shared): src/Callback.os
[66%] Compiling (shared): src/Response.os
[74%] Compiling (shared): src/Router.os
[81%] Compiling (shared): src/Route.os
[88%] Compiling (shared): src/Regex.os
[96%] Compiling (shared): src/tools.os
[100%] Linking   (shared): libexpresscpp.so
scons: done building targets.

Answer 3

我非常喜欢 RPGilespie 的回答，并在我们拥有约 4500 个节点的软件包中试用了它。不过，我做了一些调整，并认为我会发布它以防万一它对任何人有帮助。主要区别在于：

1.) 我添加了一个在作业结束时运行的命令，以将总节点数写入基于this post 的文件。这样就省去了在进度函数中不断访问文件系统的麻烦。

2.) 由于 1.) 每次运行 scons 时它都会更新文件（不仅仅是在它检测到节点数超过之前的最大值时）。如果您正在处理树的单个部分（即较少的节点）并且不断重建该部分，这将很有用。

3.) 我让它把回车放在屏幕更新之前，这样它就可以直接从 progress_function 写入它，而不是为以后设置一个变量。这有一个额外的好处，即在开始扫描一个主要构建的树时显示计数器更新。

作为旁注，我在这里将更新间隔设置为 1。原因是因为我注意到有时与单个构建命令关联的节点数少于导致它不打印该行的计数器的时间间隔。通过上述对计数器文件写入方式的更改，我没有看到任何明显的减速。 YMMV。

我将以下内容放在我的 SConstruct 文件的末尾：

screen = open('/dev/tty', 'w')
node_count = 0
node_count_max = 0
node_count_interval = 1
node_count_fname = str(env.Dir('#')) + '/.scons_node_count'

def progress_function(node):
    global node_count, node_count_max, node_count_interval, node_count_fname
    node_count += node_count_interval
    if node_count > node_count_max: node_count_max = 0
    if node_count_max>0 :
        screen.write('\r[%3d%%] ' % (node_count*100/node_count_max))
        screen.flush()

def progress_finish(target, source, env):
    global node_count
    with open(node_count_fname, 'w') as f: f.write('%d\n' % node_count)

try:
    with open(node_count_fname) as f: node_count_max = int(f.readline())
except: pass
Progress(progress_function, interval=node_count_interval)

progress_finish_command = Command('progress_finish', [], progress_finish)
Depends(progress_finish_command, BUILD_TARGETS)
if 'progress_finish' not in BUILD_TARGETS:     
    BUILD_TARGETS.append('progress_finish')

示例输出为：

[  0%] Installing [/Users/davidl/HallD/builds/sim-recon/Darwin_macosx10.11-x86_64-llvm8.0.0/bin/MakeEventWriterROOT.pl] 
[  0%] Installing [/Users/davidl/HallD/builds/sim-recon/Darwin_macosx10.11-x86_64-llvm8.0.0/bin/MakeReactionPlugin.pl] 
[  0%] Compiling  [programs/Simulation/genr8/genkin.c] 
[  0%] Compiling  [programs/Simulation/genr8/genr8.c] 
[  3%] Compiling  [programs/Utilities/hddm/hddm-cpp.cpp] 
[  3%] Compiling  [programs/Utilities/hddm/XString.cpp] 
[  3%] Compiling  [programs/Utilities/hddm/XParsers.cpp] 
[  3%] Compiling  [programs/Utilities/hddm/md5.c] 
[  4%] Compiling  [external/xstream/src/base64.cpp] 
[  4%] Compiling  [external/xstream/src/bz.cpp] 
[  4%] Compiling  [external/xstream/src/common.cpp] 
[  4%] Compiling  [external/xstream/src/dater.cpp] 
[  4%] Linking    [.Darwin_macosx10.11-x86_64-llvm8.0.0/programs/Simulation/genr8/genr8] 
[  4%] Installing [/Users/davidl/HallD/builds/sim-recon/Darwin_macosx10.11-x86_64-llvm8.0.0/bin/genr8] 
[  4%] Compiling  [external/xstream/src/debug.cpp] 
[  4%] Compiling  [external/xstream/src/digest.cpp]
 ...

为了完整起见，这里是我的 COMSTR 定义：

env.Replace(  CCCOMSTR        = "Compiling  [$SOURCE]",
              CXXCOMSTR       = "Compiling  [$SOURCE]",
              FORTRANPPCOMSTR = "Compiling  [$SOURCE]",
              FORTRANCOMSTR   = "Compiling  [$SOURCE]",
              SHCCCOMSTR      = "Compiling  [$SOURCE]",
              SHCXXCOMSTR     = "Compiling  [$SOURCE]",
              LINKCOMSTR      = "Linking    [$TARGET]",
              SHLINKCOMSTR    = "Linking    [$TARGET]",
              INSTALLSTR      = "Installing [$TARGET]",
              ARCOMSTR        = "Archiving  [$TARGET]",
              RANLIBCOMSTR    = "Ranlib     [$TARGET]")