【问题标题】:Downloading large files in Windows command prompt / PowerShell在 Windows 命令提示符/PowerShell 中下载大文件
【发布时间】:2018-03-31 12:56:33
【问题描述】:

我现在已经没有办法了……

尝试 1

在 Powershell 中使用 iwr。它可以工作,显示进度,但速度慢 10 倍,并且直到文件在内存中时才会刷新:(。

 powershell -command "& { iwr https://github.com/mitchellspryn/AirsimHighPolySuv/releases/download/V1.0.0/SUV.zip -OutFile SUV.zip }"

尝试 2

在 Powershell 中使用 .Net 网络客户端。它可以工作,但没有任何进展,你不能通过 Ctrl+C 终止:(。最后一个问题是一个很大的挫折。

powershell -command "& { (New-Object System.Net.WebClient).DownloadFile('https://github.com/mitchellspryn/AirsimHighPolySuv/releases/download/V1.0.0/SUV.zip', 'SUV.zip') }"

尝试 3

在 Powershell 中使用 BITS 传输。它可以工作,显示进度并且几乎完美...直到您发现它mysteriously doesn't work on GitHub(禁止出现 403 错误)!!

powershell -command "& { Start-BitsTransfer -Source https://github.com/mitchellspryn/AirsimHighPolySuv/releases/download/V1.0.0/SUV.zip -Destination SUV.zip }"

【问题讨论】:

  • 为什么要单独运行powershell.exe (powershell.exe -command ...)?只需直接从 PowerShell 命令行运行您需要的命令。
  • 我正在从另一个构建脚本中使用它,现在将整个脚本转换为 PS 不是我们的目标。
  • 我今天在 github 资产上尝试了Start-BitsTransfer,它成功了。 (甚至尝试过你的)。现在这有可能像你想要的那样在 Github 上运行吗?

标签: powershell command-line webclient-download microsoft-bits


【解决方案1】:

经过一番研究,我发现@MichaelS 的方法最简单,但我添加了一些你可能会觉得有用的修改,例如:

  1. 添加了Try/Finally 块,因此我们可以处理用户中断(Ctrl+C)和清理(参见Gracefully stopping in Powershell
  2. 利用Write-Progress功能,感觉更像Start-BitsTransfer
  3. 缓冲区大小优化,至少针对我的特定连接/机器(参见https://www.microsoft.com/en-us/research/wp-content/uploads/2004/12/tr-2004-136.pdf,参见https://stackoverflow.com/a/3034155/10504393

实际上它看起来像that gif

function Get-FileFromURL {
    [CmdletBinding()]
    param(
        [Parameter(Mandatory, Position = 0)]
        [System.Uri]$URL,
        [Parameter(Mandatory, Position = 1)]
        [string]$Filename
    )

    process {
        try {
            $request = [System.Net.HttpWebRequest]::Create($URL)
            $request.set_Timeout(5000) # 5 second timeout
            $response = $request.GetResponse()
            $total_bytes = $response.ContentLength
            $response_stream = $response.GetResponseStream()

            try {
                # 256KB works better on my machine for 1GB and 10GB files
                # See https://www.microsoft.com/en-us/research/wp-content/uploads/2004/12/tr-2004-136.pdf
                # Cf. https://stackoverflow.com/a/3034155/10504393
                $buffer = New-Object -TypeName byte[] -ArgumentList 256KB
                $target_stream = [System.IO.File]::Create($Filename)

                $timer = New-Object -TypeName timers.timer
                $timer.Interval = 1000 # Update progress every second
                $timer_event = Register-ObjectEvent -InputObject $timer -EventName Elapsed -Action {
                    $Global:update_progress = $true
                }
                $timer.Start()

                do {
                    $count = $response_stream.Read($buffer, 0, $buffer.length)
                    $target_stream.Write($buffer, 0, $count)
                    $downloaded_bytes = $downloaded_bytes + $count

                    if ($Global:update_progress) {
                        $percent = $downloaded_bytes / $total_bytes
                        $status = @{
                            completed  = "{0,6:p2} Completed" -f $percent
                            downloaded = "{0:n0} MB of {1:n0} MB" -f ($downloaded_bytes / 1MB), ($total_bytes / 1MB)
                            speed      = "{0,7:n0} KB/s" -f (($downloaded_bytes - $prev_downloaded_bytes) / 1KB)
                            eta        = "eta {0:hh\:mm\:ss}" -f (New-TimeSpan -Seconds (($total_bytes - $downloaded_bytes) / ($downloaded_bytes - $prev_downloaded_bytes)))
                        }
                        $progress_args = @{
                            Activity        = "Downloading $URL"
                            Status          = "$($status.completed) ($($status.downloaded)) $($status.speed) $($status.eta)"
                            PercentComplete = $percent * 100
                        }
                        Write-Progress @progress_args

                        $prev_downloaded_bytes = $downloaded_bytes
                        $Global:update_progress = $false
                    }
                } while ($count -gt 0)
            }
            finally {
                if ($timer) { $timer.Stop() }
                if ($timer_event) { Unregister-Event -SubscriptionId $timer_event.Id }
                if ($target_stream) { $target_stream.Dispose() }
                # If file exists and $count is not zero or $null, than script was interrupted by user
                if ((Test-Path $Filename) -and $count) { Remove-Item -Path $Filename }
            }
        }
        finally {
            if ($response) { $response.Dispose() }
            if ($response_stream) { $response_stream.Dispose() }
        }
    }
}

【讨论】:

    【解决方案2】:

    不知道我最初是从哪里得到这段代码的,但我已经修改了好几次。希望这会对你有所帮助。

    function downloadFile($url, $targetFile)
    {
        "Downloading $url"
        $uri = New-Object "System.Uri" "$url"
        $request = [System.Net.HttpWebRequest]::Create($uri)
        $request.set_Timeout(15000) #15 second timeout
        $response = $request.GetResponse()
        $totalLength = [System.Math]::Floor($response.get_ContentLength()/1024)
        $responseStream = $response.GetResponseStream()
        $targetStream = New-Object -TypeName System.IO.FileStream -ArgumentList $targetFile, Create
        $buffer = new-object byte[] 10KB
        $count = $responseStream.Read($buffer,0,$buffer.length)
        $downloadedBytes = $count
        while ($count -gt 0)
        {
            [System.Console]::CursorLeft = 0
            [System.Console]::Write("Downloaded {0}K of {1}K", [System.Math]::Floor($downloadedBytes/1024), $totalLength)
            $targetStream.Write($buffer, 0, $count)
            $count = $responseStream.Read($buffer,0,$buffer.length)
            $downloadedBytes = $downloadedBytes + $count
        }
        "Finished Download"
        $targetStream.Flush()
        $targetStream.Close()
        $targetStream.Dispose()
        $responseStream.Dispose()
    }
    
    downloadFile "http://URL_to_your_file" "C:\Path\to\destination.file"
    

    【讨论】:

    • 用户可以通过Ctrl+C取消吗?
    猜你喜欢
    • 1970-01-01
    • 2022-01-25
    • 1970-01-01
    • 2021-02-16
    • 1970-01-01
    • 1970-01-01
    • 2011-04-02
    • 2012-12-08
    相关资源
    最近更新 更多