从包含路径列表的 FTP 服务器下载文件并下载每个列出的文件

Question

我正在做一个项目，我需要从 ftp 服务器下载图像才能将它们放到网站上。

图像的路径位于服务器上压缩的 txt 文件中。每个图像的路径如下所示：

pc33c1_26126_08_hd.jpg /photos/pvo/transfertvo/photos/pc33c1/pc33c1_x48_08_hd.jpg 1a71ffb90de7628b8b1585b7e85c68e7

获取照片的方式应该是这样的：

get /photos/pvo/transfertvo/photos/pc33c1/pc33c1_x48_08_hd.jpg pc33c1_26126_08_hd.jpg

此示例由托管图像的公司提供。

当我使用 PowerShell 连接到 ftp 时，有两个可用目录 - 一个是 zip 文件所在的位置，另一个是空的。当您查看 .txt 文件中的路径时，您可以清楚地看到图片实际上位于第二个看似空的目录中。在PowerShell中使用get命令可以下载图片，但只能一张一张（而且有上百张图片需要下载）。

我尝试编写一个 PowerShell 脚本，该脚本遍历 .txt 文件并“获取”查找图像所需的 URL。首先我需要连接到 FTP，但是下面的代码不起作用：

$FTPServer = "ftp.example.com"
$FTPUsername = "user"
$FTPPassword = "pass"
$credential = New-Object System.Net.NetworkCredential($FTPUsername, $FTPPassword)

$FTP = [System.Net.FtpWebRequest]::Create($FTPServer)

$FTP.Credentials=$credential
$FTP.UseBinary = 1
$FTP.KeepAlive = 0

当我从 FTP 下载第二段代码后，它应该会通过 .txt 文件，但我一开始就无法连接到服务器。

$f = Get-Content "photos.txt"
$i = 0

foreach ($line in $f) {

    $fields = $line.Split("`t")

    $First = $fields[0]
    $Second = $fields[1]
    $Third = $fields[2]
    $number = $i++

    $url = $Second +" "+ $First 

    write-host "Title is: "$First
    write-host "Path is: "$Second
    write-host "Hash is: "$Third
    write-host "Number is: " $number
    Write-Host "This is the url: " $url
    get $url
    Write-Host ""
}

这应该做什么，它应该遍历数组并分割每一行以创建具有各自名称的图像的 URL。但是没有连接，它什么也做不了。

是否可以在 PowerShell FTP 中运行脚本来一次下载所有这些图像？

我也试过写一个PHP脚本，但是遇到了很多问题。代码如下：

<?php

$ftp_server = 'ftp.example.com';
$ftp_user_name = 'user';
$ftp_user_pass = 'pass';
$remoteFilePath = '/data/photos.txt.zip';
$localFilePath = $_SERVER['DOCUMENT_ROOT']."/path/";

// set up basic connection
$conn_id = ftp_connect($ftp_server);

// login with username and password
$login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);

if ((!$conn_id) || (!$login_result)) {
    echo 'FTP connection has failed! Attempted to connect to '. $ftp_server. ' for user '.$ftp_user_name.'.';
}else{
    echo 'FTP connection was a success.<br>';
    $directory = ftp_nlist($conn_id,'');
    echo '<pre>'.print_r($directory,true).'</pre>';

    $contents = ftp_nlist($conn_id, "/data/");
    var_dump($contents);

    $bla = ftp_nlist($conn_id, "/photos/");
    var_dump($bla);

    ftp_pasv($conn_id, true);

    if (ftp_get($conn_id, $localFilePath.'/photos.txt.zip', $remoteFilePath, FTP_BINARY)) {

        echo "File has been downloaded!!";
        return true;

    } else {
        echo "fail ... ";
        echo "Connected has be stopped!!";
        return false;
    }
}
ftp_close($conn_id);

?>

这会连接到 FTP，测试连接并在本地计算机上下载 zip 文件。

<?php

// assuming file.zip is in the same directory as the executing script.
$file = 'photos.txt.zip';

// get the absolute path to $file
$path = pathinfo(realpath($file), PATHINFO_DIRNAME);

$zip = new ZipArchive;
$res = $zip->open($file);
if ($res === TRUE) {
// extract it to the path we determined above
$zip->extractTo($path);
$zip->close();
echo " $file extracted to $path";
} else {
echo "Couldn't open $file";
}

?>

这个解压之前下载的 zip 文件夹在同一位置。最后一段代码尝试获取路径并下载图像，但不起作用：

header("Content-Type: text/html; charset=UTF-8");    
$lines = file("photos.txt");
$dir = "local\\path\\to\\file\\";

foreach ($lines as $line) {
    $parts = explode("\t", $line);
    $something = $parts[1];
    $somethingelse = $parts[0];
    $var1 = $ftp_server.$something;

    file_put_contents($dir, file_get_contents($var1));
}

它成功拆分数组并获取 URL，但它不下载图像而是给我错误：

file_get_contents (...) failed to open stream: No such file or directory in...

file_get_contents (...) failed to open stream: Permission denied in...

我已尝试更改其他一些主题中建议的所有权限，但无济于事。

我还发现了一个类似的主题here，但提出的解决方案已经过时并且根本没有帮助。

我必须说我对 PowerShell 和 PHP 还很陌生，如果你能提供任何帮助，我将不胜感激。

Answer 1

我最终使用了 PHP。这是工作代码：

1- 下载压缩包

$ftp_server = 'ftp.example.com';
$ftp_user_name = 'user';
$ftp_user_pass = 'psasword';
$remoteFilePath = '/data/photos.txt.zip';
$localFilePath = $_SERVER['DOCUMENT_ROOT']."/images";


// set up basic connection
$conn_id = ftp_connect($ftp_server);

// login with username and password
$login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);

if ((!$conn_id) || (!$login_result)) {
 echo 'FTP connection has failed! Attempted to connect to '. $ftp_server. ' for user '.$ftp_user_name.'.';
}else{
 echo 'FTP connection was a success.<br>';
 $directory = ftp_nlist($conn_id,'');
 echo '<pre>'.print_r($directory,true).'</pre>';

$contents = ftp_nlist($conn_id, "/data/");
var_dump($contents);

$photos = ftp_nlist($conn_id, "/photos/");
  var_dump($photos);


ftp_pasv($conn_id, true);

if (ftp_get($conn_id, $localFilePath.'/ddda-photos.txt.zip', $remoteFilePath, FTP_BINARY)) {

    echo "File has been downloaded!!";
    return true;

} else {
    echo "fail ... ";
    echo "Connected has be stopped!!";
    return false;

 }

}

2-解压文件

// assuming file.zip is in the same directory as the executing script.
$file = 'photos.txt.zip';

// get the absolute path to $file
$path = pathinfo(realpath($file), PATHINFO_DIRNAME);

$zip = new ZipArchive;
$res = $zip->open($file);
if ($res === TRUE) {
// extract it to the path we determined above
$zip->extractTo($path);
$zip->close();
echo "$file extracted to $path";
} else {
echo "Couldn't open $file";
}

3- 读取文本文件并下载图片

//So it doesn't time out
set_time_limit(0);

$ftp_server = 'ftp.example.com';
$ftp_user_name = 'user';
$ftp_user_pass = 'password';

$lines = file("photos.txt");
$dir = "F://test/";

//Goes through the file and separates name path and hash
foreach ($lines as $line) {
    $parts = explode("\t", $line);
    $imagePath = $parts[1];
    $imageName = $parts[0];
    $var1 = $ftp_server.trim($imagePath,'/');

    $ftpUrl = 'ftp://'.$ftp_user_name.':'.$ftp_user_pass.'@'.$ftp_server.'/'.$imagePath;

    // ftp url dump to be sure that something is happening
    var_dump($ftpUrl);

    $curl = curl_init();
    $fh   = fopen($dir.$imageName, 'w');
    curl_setopt($curl, CURLOPT_URL, $ftpUrl);
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
    $result = curl_exec($curl);

    fwrite($fh, $result);
    fclose($fh);
    curl_close($curl);
}

问题在于，对于每个请求，cURL 都必须传递用户名和密码才能下载图片。

希望这可以帮助遇到同样问题的人。

Answer 2

您似乎期望在使用FtpWebRequest“连接”到 FTP 后，get 命令会以某种方式神奇地从 FTP 服务器获取文件。

这假设了很多根本不正确的事情，以至于很难猜测从哪里开始解释。

• PowerShell 中没有get。这是 FTP 客户端中的常用命令。
• FtpWebRequest 不能只是连接。该课程旨在完成一次性工作，例如下载文件等。而不是创建持久连接。
• 等等等等等等。

---

基本上，您需要使用WebRequest (FtpWebRequest) 下载列表文件，然后对每个文件重复相同的操作：

$credentials = New-Object System.Net.NetworkCredential("username", "password") 

$baseUrl = "ftp://ftp.example.com"
$listPath = "/remote/path/list.txt"
$listUrl = $baseUrl + $listPath

Write-Host "Retrieving file list from $listPath..."

$listRequest = [Net.WebRequest]::Create($listUrl)
$listRequest.Method = [System.Net.WebRequestMethods+Ftp]::DownloadFile
$listRequest.Credentials = $credentials

$lines = New-Object System.Collections.ArrayList

$listResponse = $listRequest.GetResponse()
$listStream = $listResponse.GetResponseStream()
$listReader = New-Object System.IO.StreamReader($listStream)
while (!$listReader.EndOfStream)
{
    $line = $listReader.ReadLine()
    $lines.Add($line) | Out-Null
}
$listReader.Dispose()
$listStream.Dispose()
$listResponse.Dispose()

foreach ($line in $lines)
{
    $fields = $line.Split("`t")
    $file = $fields[0]
    $remoteFilePath = $fields[1]

    Write-Host "Downloading $remoteFilePath..."
    $fileUrl = $baseUrl + $remoteFilePath
    $localFilePath = $file

    $downloadRequest = [Net.WebRequest]::Create($fileUrl)
    $downloadRequest.Method = [System.Net.WebRequestMethods+Ftp]::DownloadFile
    $downloadRequest.Credentials = $credentials

    $downloadResponse = $downloadRequest.GetResponse()
    $sourceStream = $downloadResponse.GetResponseStream()
    $targetStream = [System.IO.File]::Create($localFilePath)

    $sourceStream.CopyTo($targetStream)

    $targetStream.Dispose()
    $sourceStream.Dispose()
    $downloadResponse.Dispose()
}

---

代码与我对PowerShell FTP download files and subfolders 的回答基本相同，只是它从文件中检索文件列表，而不是从目录列表中检索。