【问题标题】:Whitesource Bolt Azure DevOps plugin takes long time (and is too chatty)Whitesource Bolt Azure DevOps 插件需要很长时间(而且太健谈)
【发布时间】:2021-07-06 08:47:51
【问题描述】:

这发生在昨天之后。

在过去的几个月里,我将Whitesource Boltscan(流行的 Snyk 的免费替代品)集成到了我们的 DevOps 项目中。

扫描我们的包裹通常需要几分钟,我们对管道感到满意。

这是来自管道的典型编辑日志

Starting: WhiteSource Bolt Scan
==============================================================================
Task         : WhiteSource Bolt
Description  : Detect security vulnerabilities, problematic open source licenses.
Version      : 21.3.2
Author       : WhiteSource
Help         : http://www.whitesourcesoftware.com
==============================================================================
Working directory is /home/vsts/work/1/s
Getting scan config data
unifiedAgent.config file created successfully at /home/vsts/work/1/s
Finished getScanConfigData
Finished archive and encryption
Starting Upload zip file to s3
Getting temp credentials
Finished to prepare scm scan request
Sending SCM scan request
Succeed to send SCM scan request
WhiteSource Support Token: 
Async Command Start: Add Build Tag
Build '4998' has following tags now: ws_support_token=ws_scan_start_time=Wed, 05 May 2021 12_32_26 GMT
Async Command End: Add Build Tag
Async Command Start: Add Build Tag
Build '4998' has following tags now: ws_support_token=
Async Command End: Add Build Tag
Finishing: WhiteSource Bolt Scan

从昨天开始,输出日志爆炸变成了以下无穷无尽的调试日志,一个 Angular 项目需要 30 分钟

Starting: WhiteSource Bolt Scan
==============================================================================
Task         : WhiteSource Bolt
Description  : Detect security vulnerabilities, problematic open source licenses.
Version      : 21.6.2
Author       : WhiteSource
Help         : http://www.whitesourcesoftware.com
==============================================================================





[CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]     resolved url in file = https://pkgs.dev.azure.com/_/_packaging/_/npm/registry/@babel/plugin-transform-template-literals/-/plugin-transform-template-literals-7.13.0.tgz
[DEBUG] [2021-07-06 08:41:49,836 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://pkgs.dev.azure.com/@babel/plugin-transform-template-literals/7.13.0
[DEBUG] [2021-07-06 08:41:49,918 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined
[DEBUG] [2021-07-06 08:41:50,043 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   Succeed to download the npm package @babel/plugin-transform-modules-umd-7.13.0.tgz-7.13.0.
[DEBUG] [2021-07-06 08:41:50,043 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in file = https://pkgs.dev.azure.com/_/_packaging/_/npm/registry/@babel/plugin-transform-modules-amd/-/plugin-transform-modules-amd-7.13.0.tgz
[DEBUG] [2021-07-06 08:41:50,043 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://pkgs.dev.azure.com/@babel/plugin-transform-modules-amd/7.13.0
[DEBUG] [2021-07-06 08:41:50,085 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined
[DEBUG] [2021-07-06 08:41:50,085 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   Succeed to download the npm package @babel/plugin-syntax-optional-chaining-7.8.3.tgz-7.8.3.
[DEBUG] [2021-07-06 08:41:50,086 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in file = https://pkgs.dev.azure.com/_/_packaging/_/npm/registry/babel-plugin-dynamic-import-node/-/babel-plugin-dynamic-import-node-2.3.3.tgz
[DEBUG] [2021-07-06 08:41:50,086 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://pkgs.dev.azure.com/babel-plugin-dynamic-import-node/2.3.3
[DEBUG] [2021-07-06 08:41:50,146 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined
[DEBUG] [2021-07-06 08:41:50,147 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   Succeed to download the npm package @babel/compat-data-7.13.8.tgz-7.13.8.
[DEBUG] [2021-07-06 08:41:50,147 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in file = https://registry.npmjs.org/object.assign/-/object.assign-4.1.0.tgz
[DEBUG] [2021-07-06 08:41:50,147 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://registry.npmjs.org/object.assign/4.1.0
[DEBUG] [2021-07-06 08:41:50,256 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined
[DEBUG] [2021-07-06 08:41:50,258 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   Succeed to download the npm package @babel/plugin-proposal-logical-assignment-operators-7.13.8.tgz-7.13.8.
[DEBUG] [2021-07-06 08:41:50,258 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in file = https://pkgs.dev.azure.com/_/_packaging/_/npm/registry/@babel/plugin-transform-parameters/-/plugin-transform-parameters-7.13.0.tgz
[DEBUG] [2021-07-06 08:41:50,258 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://pkgs.dev.azure.com/@babel/plugin-transform-parameters/7.13.0
[DEBUG] [2021-07-06 08:41:51,633 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined

我们从未更改过管道配置

      - task: WhiteSource@21
        displayName: WhiteSource Bolt Scan
        inputs:
          cwd: '$(System.DefaultWorkingDirectory)'
          projectName: '$(projectName)'

有人也注意到了吗?除了为其他服务放弃这个插件之外,我们还能做些什么呢?

【问题讨论】:

  • 请注意,自 2021 年 5 月以来,该插件的版本已从 21.3.2 更改为 21.6.2,而我们没有采取任何行动
  • 这里有同样的问题。我注意到 nuget 包没有发生这种情况,因为我的 api 项目构建良好。在前端项目中,需要 45 分钟到一个小时。我认为这与 5 月的版本更改无关,因为直到昨天我的管道一直运行良好。

标签: azure-devops azure-pipelines dependency-management whitesource-bolt


【解决方案1】:

这是来自 Whitesource 支持的官方反馈

从 21.6.2 版开始,WhiteSource 扫描直接在 Azure DevOps 管道中执行。这意味着 WhiteSource 任务正在作为管道构建的一部分运行扫描。

在此更改之前,WhiteSource 任务没有直接执行扫描,它收集相关信息,将其发送到远程 WhiteSource 服务器,该服务器本身就是运行扫描的服务器。只有当远程服务器上的扫描完成并返回结果时,才会显示有关 Azure DevOps 的 WhiteSource 风险报告。这导致 WhiteSource 报告在很长一段时间后被加载并出现了几个问题。因此,我们决定对直接扫描进行更改,这是一种更直接的扫描方法,并且 WhiteSource 报告的加载速度更快,并且还有许多其他改进。但是,重要的是要了解,现在扫描是作为构建的一部分同步执行的(而不是远程异步),与以前的版本相比,构建时间(而不是扫描时间)增加了。

所以看起来他们在没有警告用户管道将花费更长时间的情况下推动了重大变化

【讨论】:

  • 哇哦!不需要更长的时间!涉及 npm 包需要很长时间。有一个 Owasp 依赖检查插件。我会试一试。其他可能的解决方案:缓存 npm 包?仔细检查开发依赖项是否不是检查的一部分?
  • 我会和我的老板谈谈购买 Snyk 的事情。我刚刚从我们所有项目中删除了所有 WS 管道任务
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2017-01-18
  • 1970-01-01
  • 2021-07-24
  • 2016-08-13
  • 2017-12-05
  • 2012-07-17
  • 1970-01-01
相关资源
最近更新 更多