【问题标题】:Why doesn't VSTS build agent download actual Git LFS file but only metadata?为什么 VSTS 构建代理不下载实际的 Git LFS 文件而只下载元数据?
【发布时间】:2017-10-13 19:31:17
【问题描述】:

我正在尝试获取存储在 Git LFS 中的新配置的 VSTS 私有构建代理下载文件,但它似乎只下载每个文件的元数据,文件内容最终如下:

version https://git-lfs.github.com/spec/v1 
oid sha256:1c4fa4fba8880241858d324b17b6b95512b65f98ea3ea1e959659f0176e45ec2 
size 23552

VSTS 构建代理的版本为 2.123.0。

为了让 Git LFS 完全正常工作,请记住这是在内部服务器上运行的私有代理,我必须手动安装它,它的版本为 2.3.3。

代理正在运行以下命令;

初始化:

2017-10-11T13:19:25.8153872Z ##[command]git version
2017-10-11T13:19:25.9089878Z git version 2.12.2.windows.3
2017-10-11T13:19:25.9245879Z ##[command]git lfs version
2017-10-11T13:19:26.1749886Z git-lfs/2.3.3 (GitHub; windows amd64; go 1.8.3; git c9d3beb0)
2017-10-11T13:19:26.1749886Z ##[command]git config --get remote.origin.url
2017-10-11T13:19:26.2529886Z ##[command]git config gc.auto 0
2017-10-11T13:19:26.2997886Z ##[command]git config --get-all http.https://aaa.visualstudio.com/bbb/_git/ccc.extraheader
2017-10-11T13:19:26.3621886Z ##[command]git config --get-all http.proxy
2017-10-11T13:19:26.4089886Z ##[command]git lfs install --local
2017-10-11T13:19:27.2705886Z Updated git hooks.
2017-10-11T13:19:27.2705886Z Git LFS initialized.

主 Git 获取命令:

2017-10-11T13:19:27.3027886Z ##[command]git -c http.extraheader="AUTHORIZATION: bearer ********" fetch --tags --prune --progress --no-recurse-submodules origin
[...]
2017-10-11T13:19:29.7987886Z Receiving objects:   0% (1/149024)   

Git LFS 获取命令:

2017-10-11T13:21:24.5211886Z ##[command]git -c http.extraheader="AUTHORIZATION: bearer ********" lfs fetch origin 3893c0f7d8847adc21182dfd6d10c5d065bda565
2017-10-11T13:21:24.7745886Z Fetching 3893c0f7d8847adc21182dfd6d10c5d065bda565
2017-10-11T13:21:25.9157886Z ##[command]git checkout --progress --force 3893c0f7d8847adc21182dfd6d10c5d065bda565
2017-10-11T13:21:27.1799886Z Checking out files:   4% (550/11027)   

最终子模块(LFS 文件实际驻留的位置)同步:

2017-10-11T13:21:47.8469886Z ##[command]git submodule sync
2017-10-11T13:21:49.2977886Z ##[command]git -c http.https://aaa.visualstudio.com.extraheader="AUTHORIZATION: bearer ********" submodule update --init --force
2017-10-11T13:21:49.9997886Z Submodule 'ExternalSoftware' (https://aaa.visualstudio.com/bbb/_git/ccc.ExternalSoftware) registered for path 'ExternalSoftware'
2017-10-11T13:21:51.5285886Z Cloning into 'E:/agent/_work/1/s/ExternalSoftware'...
2017-10-11T13:21:53.3537886Z Submodule path 'ExternalSoftware': checked out '71ce8e95c4f43447c1d1419be7ffb7829a517c2f'
2017-10-11T13:21:53.3693886Z ##[section]Finishing: Get Sources

那么,我做错了什么只获取基于 LFS 的文件的元数据,而不是实际的文件内容?

似乎我遇到的问题与Check out git-lfs files on VSTF build 略有相关,但仍然通过我使用的相同的开箱即用 Git LFS 支持解决了该问题。


更新:
作为测试,我在另一台服务器上尝试了 VSTS 代理,它能够下载文件。
也就是我没有服务器的登录权限,所以没有验证文件内容,但是记录了以下内容:

2017-10-12T07:44:26.4608720Z ##[command]git -c http.extraheader="AUTHORIZATION: bearer ********" lfs pull
2017-10-12T07:44:29.2051354Z 
2017-10-12T07:44:29.4005140Z Git LFS: (0 of 201 files) 0 B / 36.67 MB                                       
2017-10-12T07:44:29.6005312Z Git LFS: (0 of 201 files) 0 B / 36.67 MB                                       
2017-10-12T07:44:29.8021911Z Git LFS: (0 of 201 files) 0 B / 36.67 MB                                       
2017-10-12T07:44:30.0024064Z Git LFS: (0 of 201 files) 256.00 KB / 36.67 MB                                 
2017-10-12T07:44:30.2025273Z Git LFS: (0 of 201 files) 960.00 KB / 36.67 MB                                 
2017-10-12T07:44:30.4029958Z Git LFS: (0 of 201 files) 1.38 MB / 36.67 MB                                   
2017-10-12T07:44:30.6039364Z Git LFS: (0 of 201 files) 1.62 MB / 36.67 MB                                   
2017-10-12T07:44:30.8049023Z Git LFS: (1 of 201 files) 1.92 MB / 36.67 MB                                   
[...]
2017-10-12T07:45:10.4673396Z Git LFS: (275 of 275 files) 103.98 MB / 103.98 MB                              
2017-10-12T07:45:10.4943356Z ##[section]Finishing: Pull external software LFS files

由于其他第三方“功能”的原因,我还不能使用那个代理,但它证明该机制应该能够为我的存储库工作。


更新:
@Marina-MSFT 请求了完整的日志,所以这里是(它来自最近的运行,所以时间戳与上面有点不同):

2017-10-13T10:26:07.0921581Z ##[section]Starting: Get Sources
2017-10-13T10:26:07.1389599Z Syncing repository: ccc (TfsGit)
2017-10-13T10:26:07.1545605Z Prepending Path environment variable with directory containing 'git.exe'.
2017-10-13T10:26:07.1545605Z ##[command]git version
2017-10-13T10:26:07.2481641Z git version 2.12.2.windows.3
2017-10-13T10:26:07.2637647Z ##[command]git lfs version
2017-10-13T10:26:07.3271672Z git-lfs/2.3.3 (GitHub; windows amd64; go 1.8.3; git c9d3beb0)
2017-10-13T10:26:07.3583684Z ##[command]git config --get remote.origin.url
2017-10-13T10:26:07.4363714Z ##[command]git config gc.auto 0
2017-10-13T10:26:07.4987738Z ##[command]git config --get-all http.https://aaa.visualstudio.com/bbb/_git/ccc.extraheader
2017-10-13T10:26:07.5923774Z ##[command]git config --get-all http.proxy
2017-10-13T10:26:07.6859810Z ##[command]git lfs install --local
2017-10-13T10:26:08.2328280Z Updated git hooks.
2017-10-13T10:26:08.2338281Z Git LFS initialized.
2017-10-13T10:26:08.2650293Z ##[command]git -c http.extraheader="AUTHORIZATION: bearer ********" fetch --tags --prune --progress --no-recurse-submodules origin
2017-10-13T10:26:08.9358551Z ##[command]git -c http.extraheader="AUTHORIZATION: bearer ********" lfs fetch origin 440cb61e73a938c4c83da56823b53d13198ca9b9
2017-10-13T10:26:09.2744832Z Fetching 440cb61e73a938c4c83da56823b53d13198ca9b9
2017-10-13T10:26:11.4949033Z ##[command]git checkout --progress --force 440cb61e73a938c4c83da56823b53d13198ca9b9
2017-10-13T10:26:11.8381165Z HEAD is now at 440cb61e7... [some commit]
2017-10-13T10:26:12.8444104Z ##[command]git submodule sync --recursive
2017-10-13T10:26:14.2328638Z Synchronizing submodule url for 'ExternalSoftware'
2017-10-13T10:26:14.8100860Z ##[command]git -c http.https://aaa.visualstudio.com.extraheader="AUTHORIZATION: bearer ********" submodule update --init --force --recursive
2017-10-13T10:26:17.0409718Z Submodule path 'ExternalSoftware': checked out '71ce8e95c4f43447c1d1419be7ffb7829a517c2f'
2017-10-13T10:26:17.3529838Z ##[command]git config http.https://aaa.visualstudio.com/bbb/_git/ccc.extraheader "AUTHORIZATION: bearer ********"
2017-10-13T10:26:17.3997856Z ##[command]git config http.https://aaa.visualstudio.com/bbb/_git/ccc.extraheader "AUTHORIZATION: bearer ********"
2017-10-13T10:26:17.4621880Z ##[section]Finishing: Get Sources

2017-10-13T10:26:17.4621880Z ##[section]Starting: Pull external software LFS files
2017-10-13T10:26:17.4933892Z ==============================================================================
2017-10-13T10:26:17.4933892Z Task         : Command Line
2017-10-13T10:26:17.4933892Z Description  : Run a command line with arguments
2017-10-13T10:26:17.4933892Z Version      : 1.1.3
2017-10-13T10:26:17.4933892Z Author       : Microsoft Corporation
2017-10-13T10:26:17.4933892Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613735)
2017-10-13T10:26:17.4933892Z ==============================================================================
2017-10-13T10:26:17.5713922Z ##[command]git -c http.extraheader="AUTHORIZATION: bearer ********" lfs pull
2017-10-13T10:26:18.3030596Z 
2017-10-13T10:26:18.4840777Z Git LFS: (0 of 275 files) 0 B / 103.98 MB                                      
2017-10-13T10:26:18.6790972Z Git LFS: (0 of 275 files) 0 B / 103.98 MB                                      
[...]
2017-10-13T10:26:33.2805572Z Git LFS: (0 of 275 files) 0 B / 103.98 MB                                      
2017-10-13T10:26:33.3877660Z ##[error]^C

2017-10-13T10:26:33.3877660Z Git LFS: (0 of 275 files) 0 B / 103.98 MB                                      
2017-10-13T10:26:33.3877660Z ##[error]The operation was canceled.
2017-10-13T10:26:33.4033666Z ##[section]Finishing: Pull external software LFS files

【问题讨论】:

  • 这两个提交 1c4fa4f71ce8e9 是否都用于子模块 repo ExternalSoftware?哪个提交更新?如果您没有更新主仓库中的子模块,请更新。您能否显示获取源步骤的完整日志?
  • @Marina-MSFT,“1c4fa4f”不是提交,AFAIK,而是 LFS 文件之一的 sha256 的开始。他们都是不同的。但是,您提到的“71ce8e9”是主仓库中更新子模块的提交。代理已经根据原始帖子中现在包含的完整日志中显示的命令更新了子模块。
  • 我可以在获取源步骤中检查子模块及其 lfs 文件是否成功。首先请通过在本地克隆 repo 并使用git lfs ls-files 来检查子模块 repo 是否有 lfs 文件。然后请确保选择了 Checkout submodulesCheckout files from LFS 选项。我的详细日志在这里1drv.ms/t/s!ApIKkY1MTjMRgmn8Kj6b7ZrFBxlh
  • @MortenGrøtan 你设法解决了这个问题吗?我在本地 TFS 构建代理上运行同样的问题。我在父仓库中获得 LFS 文件,但在子模块中没有。

标签: git azure-devops git-lfs azure-devops-self-hosted-agent


【解决方案1】:

基本上,它是克隆子模块但绕过子模块的 LFS。多年来,GitHub 上一直在讨论它应该如何工作。

尝试添加Command Line 任务并运行git lfs pull --recursive 作为构建的第一步。

【讨论】:

  • 没有这样的运气:2017-10-11T15:39:17.1120000Z ##[command]git lfs pull --recursive 2017-10-11T15:39:17.2066019Z 2017-10-11T15:39 :17.2066019Z git lfs pull [options] [] 2017-10-11T15:39:17.2066019Z 错误:未知标志:--recursive
  • 如果我在没有“--recursive”标志的情况下运行,它不会失败,但也不会做任何事情:2017-10-11T15:49:06.0093960Z ##[command]git lfs pull 2017-10-11T15:49:08.5354369Z ##[section]整理:正确获取 Git LFS
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2019-08-08
  • 2016-10-10
  • 2019-02-12
相关资源
最近更新 更多