为什么 PowerShell 会在 stderr 上截断消息？

Question

我正在使用 PowerShell 脚本来控制编译器 (ghdl.exe) 的不同编译步骤。

编译器有 3 种不同的输出格式：

1. 没有输出也没有错误 => $LastExitCode = 0
2. stderr 上的输出（警告），但没有错误 => $LastExitCode = 0
3. stderr 上的输出（错误），可能还有警告 => $LastExitCode != 0

由于处理 stderr 和 stdout 接缝非常有问题，我使用了这篇 StackOverflow 帖子中介绍的方法：PowerShell: Manage errors with Invoke-Expression

这是我的附加消息着色的实现：

function Format-NativeCommandStreams
{ param([Parameter(ValueFromPipeline=$true)]$InputObject)

  begin
  { $ErrorRecordFound = $false  }

  process
  { if (-not $InputObject)
    { Write-Host "Empty"  }
    elseif ($InputObject -is [System.Management.Automation.ErrorRecord])
    { $ErrorRecordFound  = $true
      $text = $InputObject.ToString()
      Write-Host $text -ForegroundColor  Gray

      $stdErr = $InputObject.TargetObject
      if ($stdErr)
      { #Write-Host ("err: type=" + $stdErr.GetType() + "  " + $stdErr)
        if ($stdErr.Contains("warning"))
        { Write-Host "WARNING: "  -NoNewline -ForegroundColor Yellow  }
        else
        { Write-Host "ERROR: "    -NoNewline -ForegroundColor Red      }
        Write-Host $stdErr
      }
    }
    else
    { $stdOut = $InputObject                
      if ($stdOut.Contains("warning"))
      { Write-Host "WARNING: "  -NoNewline -ForegroundColor Yellow  }
      else
      { Write-Host "ERROR: "    -NoNewline -ForegroundColor Red      }
      Write-Host $stdOut
    }
  }

  end
  { $ErrorRecordFound    }
}

用法：

$Options = @(.....)
$Expr = "ghdl.exe -a " + ($Options -join " ") + " " + $File + " 2>&1"
$ret = Invoke-Expression $Expr | Format-NativeCommandStreams

通常，编译器每行发出一条消息（错误或警告）。如下图所示，一些消息被截断为 8 行。这就是为什么我的输出着色不能按预期工作的原因。更多的行被检测为错误（误报），所以我在日志中找不到真正的错误。

（可点击）

示例：

C:\Altera\15.0\quartus\eda\sim_lib\altera_mf.vhd:
39963:
53
:
warning:

universal integer bound must be numeric literal or attribute

C:\Altera\15.0\quartus\eda\sim_lib\altera_mf.vhd
:41794:36:warning: universal integer bound must be numeric literal or attribute

预期结果：

C:\Altera\15.0\quartus\eda\sim_lib\altera_mf.vhd:39963:53:warning: universal integer bound must be numeric literal or attribute
C:\Altera\15.0\quartus\eda\sim_lib\altera_mf.vhd:41794:36:warning: universal integer bound must be numeric literal or attribute

据我所知，编译器 (ghdl.exe) 确实将消息作为整行发出。

问题：

• 为什么会这样？
• 谁能解决这个问题？

Answer 1

解决方案

可执行文件的stderr 上的完整输出简单地拆分为多个System.Management.Automation.ErrorRecord 类型的对象。实际的拆分似乎是不确定的(*)。此外，部分字符串存储在属性Exception 中，而不是TargetObject。只有第一个 ErrorRecord 具有非空 TargetObject。也就是说，为什么包含字符串 "warning" 的输出的后续行没有以黄色和白色格式设置，就像这样：

:41794:36:warning: universal integer bound must be numeric literal or attribute

您的灰色输出来自每个ErrorRecord 的toString() 方法，该方法返回此记录的属性Exception.Message 的值。 因此必须在格式化之前将所有消息连接在一起以获得整个输出。这些消息中会保留换行符。

编辑：(*) 这取决于程序的写入/刷新调用相对于 Powershell 的读取调用的顺序。如果在下面我的测试程序中的每个fprintf() 之后添加fflush(stderr)，将会有更多ErrorRecord 对象。除了第一个似乎是确定性的，其中一些包括 2 条输出线，其中一些包括 3 条。

我的测试平台

我没有使用 GHDL，而是从一个新的 Visual Studio 项目开始，并使用以下代码创建了一个控制台应用程序 (HelloWorldEx)。它只是在stderr 上打印出很多编号的行

#include "stdafx.h"
#include <stdio.h>

int _tmain(int argc, _TCHAR* argv[])
{
  // Print some warning messages on stderr
  for(int i=0; i<70; i++) {
    fprintf(stderr, "warning:%070d\n", i); // 80 bytes per line including CR+LF
  }
  return 0; // exit code is not relevant
}

然后我编译了程序并在 Powershell 中执行它： （编辑：从我自己的脚本中删除了调试代码）

.\HelloWorldEx.exe 2>&1 | set-variable Output
$i = 0
$Output | % {
  Write-Host ("--- " + $i + ": "  + $_.GetType() +  " ------------------------")
  Write-Host ($_ | Format-List -Force | Out-String)
  $i++
}

这是脚本的输出。如您所见，我的程序的输出被拆分为 3 ErrorRecords（实际可能不同）：

--- 0: System.Management.Automation.ErrorRecord ------------------------


writeErrorStream      : True
Exception             : System.Management.Automation.RemoteException: warning:00000000000000000000000000000000000000000
                        00000000000000000000000000000
TargetObject          : warning:0000000000000000000000000000000000000000000000000000000000000000000000
CategoryInfo          : NotSpecified: (warning:0000000...000000000000000:String) [], RemoteException
FullyQualifiedErrorId : NativeCommandError
ErrorDetails          : 
InvocationInfo        : System.Management.Automation.InvocationInfo
PipelineIterationInfo : {0, 0, 0}
PSMessageDetails      : 




--- 1: System.Management.Automation.ErrorRecord ------------------------


writeErrorStream      : True
Exception             : System.Management.Automation.RemoteException: warning:00000000000000000000000000000000000000000
                        00000000000000000000000000001
                        warning:0000000000000000000000000000000000000000000000000000000000000000000002
                        warning:0000000000000000000000000000000000000000000000000000000000000000000003
                        warning:0000000000000000000000000000000000000000000000000000000000000000000004
                        warning:0000000000000000000000000000000000000000000000000000000000000000000005
                        warning:0000000000000000000000000000000000000000000000000000000000000000000006
                        warning:0000000000000000000000000000000000000000000000000000000000000000000007
                        warning:0000000000000000000000000000000000000000000000000000000000000000000008
                        warning:0000000000000000000000000000000000000000000000000000000000000000000009
                        warning:0000000000000000000000000000000000000000000000000000000000000000000010
                        warning:0000000000000000000000000000000000000000000000000000000000000000000011
                        warning:0000000000000000000000000000000000000000000000000000000000000000000012
                        warning:0000000000000000000000000000000000000000000000000000000000000000000013
                        warning:0000000000000000000000000000000000000000000000000000000000000000000014
                        warning:0000000000000000000000000000000000000000000000000000000000000000000015
                        warning:0000000000000000000000000000000000000000000000000000000000000000000016
                        warning:0000000000000000000000000000000000000000000000000000000000000000000017
                        warning:0000000000000000000000000000000000000000000000000000000000000000000018
                        warning:0000000000000000000000000000000000000000000000000000000000000000000019
                        warning:0000000000000000000000000000000000000000000000000000000000000000000020
                        warning:0000000000000000000000000000000000000000000000000000000000000000000021
                        warning:0000000000000000000000000000000000000000000000000000000000000000000022
                        warning:0000000000000000000000000000000000000000000000000000000000000000000023
                        warning:0000000000000000000000000000000000000000000000000000000000000000000024
                        warning:0000000000000000000000000000000000000000000000000000000000000000000025
                        warning:0000000000000000000000000000000000000000000000000000000000000000000026
                        warning:0000000000000000000000000000000000000000000000000000000000000000000027
                        warning:0000000000000000000000000000000000000000000000000000000000000000000028
                        warning:0000000000000000000000000000000000000000000000000000000000000000000029
                        warning:0000000000000000000000000000000000000000000000000000000000000000000030
                        warning:0000000000000000000000000000000000000000000000000000000000000000000031
                        warning:0000000000000000000000000000000000000000000000000000000000000000000032
                        warning:0000000000000000000000000000000000000000000000000000000000000000000033
                        warning:0000000000000000000000000000000000000000000000000000000000000000000034
                        warning:0000000000000000000000000000000000000000000000000000000000000000000035
                        warning:0000000000000000000000000000000000000000000000000000000000000000000036
                        warning:0000000000000000000000000000000000000000000000000000000000000000000037
                        warning:0000000000000000000000000000000000000000000000000000000000000000000038
                        warning:0000000000000000000000000000000000000000000000000000000000000000000039
                        warning:0000000000000000000000000000000000000000000000000000000000000000000040
                        warning:0000000000000000000000000000000000000000000000000000000000000000000041
                        warning:0000000000000000000000000000000000000000000000000000000000000000000042
                        warning:0000000000000000000000000000000000000000000000000000000000000000000043
                        warning:0000000000000000000000000000000000000000000000000000000000000000000044
                        warning:0000000000000000000000000000000000000000000000000000000000000000000045
                        warning:0000000000000000000000000000000000000000000000000000000000000000000046
                        warning:0000000000000000000000000000000000000000000000000000000000000000000047
                        warning:0000000000000000000000000000000000000000000000000000000000000000000048
                        warning:0000000000000000000000000000000000000000000000000000000000000000000049
                        warning:0000000000000000000000000000000000000000000000000000000000000000000050
                        warning:00000000000000000000000000000000000000000000000000000000000
TargetObject          : 
CategoryInfo          : NotSpecified: (:) [], RemoteException
FullyQualifiedErrorId : NativeCommandErrorMessage
ErrorDetails          : 
InvocationInfo        : System.Management.Automation.InvocationInfo
PipelineIterationInfo : {0, 0, 1}
PSMessageDetails      : 




--- 2: System.Management.Automation.ErrorRecord ------------------------


writeErrorStream      : True
Exception             : System.Management.Automation.RemoteException: 00000000051
                        warning:0000000000000000000000000000000000000000000000000000000000000000000052
                        warning:0000000000000000000000000000000000000000000000000000000000000000000053
                        warning:0000000000000000000000000000000000000000000000000000000000000000000054
                        warning:0000000000000000000000000000000000000000000000000000000000000000000055
                        warning:0000000000000000000000000000000000000000000000000000000000000000000056
                        warning:0000000000000000000000000000000000000000000000000000000000000000000057
                        warning:0000000000000000000000000000000000000000000000000000000000000000000058
                        warning:0000000000000000000000000000000000000000000000000000000000000000000059
                        warning:0000000000000000000000000000000000000000000000000000000000000000000060
                        warning:0000000000000000000000000000000000000000000000000000000000000000000061
                        warning:0000000000000000000000000000000000000000000000000000000000000000000062
                        warning:0000000000000000000000000000000000000000000000000000000000000000000063
                        warning:0000000000000000000000000000000000000000000000000000000000000000000064
                        warning:0000000000000000000000000000000000000000000000000000000000000000000065
                        warning:0000000000000000000000000000000000000000000000000000000000000000000066
                        warning:0000000000000000000000000000000000000000000000000000000000000000000067
                        warning:0000000000000000000000000000000000000000000000000000000000000000000068
                        warning:0000000000000000000000000000000000000000000000000000000000000000000069

TargetObject          : 
CategoryInfo          : NotSpecified: (:) [], RemoteException
FullyQualifiedErrorId : NativeCommandErrorMessage
ErrorDetails          : 
InvocationInfo        : System.Management.Automation.InvocationInfo
PipelineIterationInfo : {0, 0, 2}
PSMessageDetails      : 

Answer 2

您可以进行一些调试来解决这个问题。我建议从这样的开始：

ghdl.exe <whatever args you supply> 2>&1 | set-variable ghdlOutput
$i = 0
$ghdlOutput | % {write-host "$i `t: " $_.gettype() "`t" $_ ; $i++}

这将列出行号、输出行的类型以及输出的每个活动。您可能需要对代码进行一些调整才能使输出看起来正常。

从那里您可以查看编译器是否真的将错误拆分为多行。如果是，您可以尝试设计一种策略来确定哪些行是标准输出，哪些是标准错误。如果没有，那么您将有一些线索来调试上面的脚本。

或者可以打包整个方法并使用 .NET system.diagnostics.process 类并将 stdout 和 stderr 重定向为单独的流。使用采用 ProcessStartInfo 的 Start 方法。如果需要，您应该能够在谷歌上搜索这样做的示例。

Answer 3

为了完整起见，这里是我当前的 CommandLets，它将错误消息恢复为单行并根据需要为它们着色：

用法：

$InvokeExpr = "ghdl.exe " + ($Options -join " ") + " --work=unisim " + $File.FullName + " 2>&1"
$ErrorRecordFound = Invoke-Expression $InvokeExpr | Collect-NativeCommandStream | Write-ColoredGHDLLine

CommandLet 恢复错误信息：

function Collect-NativeCommandStream
{ [CmdletBinding()]
  param([Parameter(ValueFromPipeline=$true)]$InputObject)

  begin
  { $LineRemainer = ""  }

  process
  { if (-not $InputObject)
    { Write-Host "Empty pipeline!"  }
    elseif ($InputObject -is [System.Management.Automation.ErrorRecord])
    { if ($InputObject.FullyQualifiedErrorId -eq "NativeCommandError")
      { Write-Output $InputObject.ToString()    }
      elseif ($InputObject.FullyQualifiedErrorId -eq "NativeCommandErrorMessage")
      { $NewLine = $LineRemainer + $InputObject.ToString()
        while (($NewLinePos = $NewLine.IndexOf("`n")) -ne -1)
        { Write-Output $NewLine.Substring(0, $NewLinePos)
          $NewLine = $NewLine.Substring($NewLinePos + 1)
        }
        $LineRemainer = $NewLine
      }
    }
    elseif ($InputObject -is [String])
    { Write-Output $InputObject    }
    else
    { Write-Host "Unsupported object in pipeline stream"    }
  }

  end
  {   }
}

CommandLet 为警告和错误着色：

function Write-ColoredGHDLLine
{ [CmdletBinding()]
  param([Parameter(ValueFromPipeline=$true)]$InputObject)

  begin
  { $ErrorRecordFound = $false  }

  process
  { if (-not $InputObject)
    { Write-Host "Empty pipeline!"  }
    elseif ($InputObject -is [String])
    { if ($InputObject.Contains("warning"))
      { Write-Host "WARNING: "  -NoNewline -ForegroundColor Yellow  }
      else
      { $ErrorRecordFound  = $true
        Write-Host "ERROR: "    -NoNewline -ForegroundColor Red
      }
      Write-Host $InputObject
    }
    else
    { Write-Host "Unsupported object in pipeline stream"    }
  }

  end
  { $ErrorRecordFound    }
}

Answer 4

似乎我设法通过 Martin Zabel 示例解决了这个问题，结果证明解决方案非常平淡无奇。

事实上，很长一段时间我都无法从来电中获取字符 `r`n。结果很简单。

将 `r 替换为 `n 是唯一需要做的事情！

该解决方案适用于任何宽度的控制台，因为已删除反向。

另外，这可能是解决将处理后的数据实时返回到控制台的问题的基础。唯一需要的是捕获传入的单个 `r 或 `n 以获取新的“变量字符串”，并将处理后的数据使用 `r 或 `n 发送回控制台，具体取决于任务。

cls
function GetAnsVal {
    param([Parameter(Mandatory=$true, ValueFromPipeline=$true)][System.Object[]][AllowEmptyString()]$Output,
          [Parameter(Mandatory=$false, ValueFromPipeline=$true)][System.String]$firstEncNew="UTF-8",
          [Parameter(Mandatory=$false, ValueFromPipeline=$true)][System.String]$secondEncNew="CP866"
    )
    function ConvertTo-Encoding ([string]$From, [string]$To){#"UTF-8" "CP866" "ASCII" "windows-1251"
        Begin{
            $encFrom = [System.Text.Encoding]::GetEncoding($from)
            $encTo = [System.Text.Encoding]::GetEncoding($to)
        }
        Process{
            $Text=($_).ToString()
            $bytes = $encTo.GetBytes($Text)
            $bytes = [System.Text.Encoding]::Convert($encFrom, $encTo, $bytes)
            $encTo.GetString($bytes)
        }
    }
    $all = New-Object System.Collections.Generic.List[System.Object];
    $exception = New-Object System.Collections.Generic.List[System.Object];
    $stderr = New-Object System.Collections.Generic.List[System.Object];
    $stdout = New-Object System.Collections.Generic.List[System.Object]
    $i = 0;$Output | % {
        if ($_ -ne $null){
            if ($_.GetType().FullName -ne 'System.Management.Automation.ErrorRecord'){
                if ($_.Exception.message -ne $null){$Temp=$_.Exception.message | ConvertTo-Encoding $firstEncNew $secondEncNew;$all.Add($Temp);$exception.Add($Temp)}
                elseif ($_ -ne $null){$Temp=$_ | ConvertTo-Encoding $firstEncNew $secondEncNew;$all.Add($Temp);$stdout.Add($Temp)}
            } else {
                #if (MyNonTerminatingError.Exception is AccessDeniedException)
                $Temp=$_.Exception.message | ConvertTo-Encoding $firstEncNew $secondEncNew;
                $all.Add($Temp);$stderr.Add($Temp)
            }   
         }
    $i++
    }
    [hashtable]$return = @{}
    $return.Meta0=$all;$return.Meta1=$exception;$return.Meta2=$stderr;$return.Meta3=$stdout;
    return $return
}
Add-Type -AssemblyName System.Windows.Forms;
& C:\Windows\System32\curl.exe 'api.ipify.org/?format=plain' 2>&1 | set-variable Output;
$r = & GetAnsVal $Output
$Meta0=""
foreach ($el in $r.Meta0){
    $Meta0+=$el
}
$Meta0=($Meta0 -split "[`r`n]") -join "`n"
$Meta0=($Meta0 -split "[`n]{2,}") -join "`n"
[Console]::Write($Meta0);
[Console]::Write("`n");