【问题标题】:How to remove nth column from the text table in powershell?如何从powershell中的文本表中删除第n列?
【发布时间】:2019-12-31 08:42:31
【问题描述】:

假设我正在使用格式良好的表格。以 kubectl 输出为例:

NAME            READY   STATUS    RESTARTS   AGE     IP          NODE   NOMINATED NODE   READINESS GATES
me-pod-name     2/2     Running   0          6s      10.0.0.10   node1  <none>           <none>
me-pod-name-2   1/1     Running   0          6d18h   10.0.1.20   node2  <none>           <none>
me-pod-name-3   1/1     Running   0          11d     10.0.0.30   node3  <none>           <none>

我倾向于观察这样的输出并记录很多变化。在这种情况下,我想从表中删除中间列之一,并且仍然得到一个不错的输出。例如。让我们尝试删除 AGE 列,因为它变化很大,并且对于资源年轻时的观看无用:

NAME            READY   STATUS    RESTARTS   IP          NODE   NOMINATED NODE   READINESS GATES
me-pod-name     2/2     Running   0          10.0.0.10   node1  <none>           <none>
me-pod-name-2   1/1     Running   0          10.0.1.20   node2  <none>           <none>
me-pod-name-3   1/1     Running   0          10.0.0.30   node3  <none>           <none>

我的问题是:如何轻松删除此类列并输出格式良好的表格,而所有其他列都完好无损?这些列的大小并不总是相同(例如,年龄并不总是 8 个字符宽的)。我想找到一些可重用的单行解决方案,因为我经常使用 CLI 工具(不仅与 k8s 相关)并且需要过滤它们。另外,我想避免基于模式的解决方案,我需要为要删除的每一列生成正则表达式 - 我能够做到,但这需要我为每个用例编写特定的解决方案。

我尝试使用 ConvertFrom-String 和格式表,但这对数据格式造成了很大影响(例如,“1/1”被视为日期格式,这不适用于这种情况)。

【问题讨论】:

    标签: powershell


    【解决方案1】:

    您正在查看的是固定宽度数据。 为了解决这个问题,Import-Csv 不会这样做,所以我前段时间做了几个函数来转换固定宽度数据。

    function ConvertFrom-FixedWidth {
        [CmdletBinding()]
        [OutputType([PSObject[]])]
        Param(
            [Parameter(Mandatory = $true, Position = 0, ValueFromPipeline = $true)]
            [string[]]$Data,
    
            [int[]]$ColumnWidths = $null,    # an array of integers containing the width in characters for each field
            [string[]]$Header = $null,       # if given, a string array containing the columnheaders
            [switch]$AllowRaggedContent      # if set, the function accepts the last items to be trimmed.
        )
    
        # If the data is sent through the pipeline, use $input to collect is as array
        if ($PSCmdlet.MyInvocation.ExpectingInput) { $Data = @($Input) }
        # or use : $Data = $Input | ForEach-Object { $_ }
    
        if (!$ColumnWidths) {
            Write-Verbose "Calculating column widths using first row"
            # Try and determine the width of each field from the top (header) line.
            # This can only work correctly if the fields in that line do not contain space
            # characters OR if each field is separated from the next by more than 1 space.
    
            # temporarily replace single spaces in the header row with underscore
            $row1 = $Data[0] -replace '(\S+) (\S+)', '$1_$2'  
    
            # Get the starting index of each field and add the total length for the last field
            $indices = @(([regex] '\S+').Matches($row1) | ForEach-Object {$_.Index}) + $row1.Length
            # Calculate individual field widths from their index
            $ColumnWidths = (0..($indices.Count -2)) | ForEach-Object { $indices[$_ + 1] - $indices[$_] }
        }
    
        # Combine the field widths integer array into a regex string like '^(.{10})(.{50})(.{12})'
        $values = $ColumnWidths | ForEach-Object { "(.{$_})" }
        if ($AllowRaggedContent) {
            # account for fields that are too short (possibly by trimming trailing spaces)
            # set the last item to be '(.*)$' to capture any characters left in the string.
            $values[-1] = '(.*)$'
        }
        $regex = '^{0}' -f ($values -join '')
    
        Write-Verbose "Splitting fields and generating output"
        # Execute a scriptblock to convert each line in the array.
        $csv = & { 
            switch -Regex ($Data) {
                $regex {
                    # Join what the capture groups matched with a comma and wrap the fields
                    # between double-quotes. Double-quotes inside a fields value must be doubled.
                    ($matches[1..($matches.Count - 1)] | ForEach-Object { '"{0}"' -f ($_.Trim() -replace '"', '""') }) -join ','
                }
            }
        }
        if ($Header) { $csv | ConvertFrom-Csv -Header $Header }
        else { $csv | ConvertFrom-Csv }
    }
    
    function ConvertTo-FixedWidth {
        [CmdletBinding()]
        [OutputType([String[]])]
        Param (
            [Parameter(Mandatory = $true, Position = 0, ValueFromPipeline = $true)]
            [PSObject[]]$Data,
    
            [Parameter(Mandatory = $false)]
            [ValidateRange(1, 8)]
            [int]$Gap = 2
        )
    
        # if the data is sent through the pipeline, use $input to collect is as array
        if ($PSCmdlet.MyInvocation.ExpectingInput) { $Data = @($Input) }
        # or use : $Data = $Input | ForEach-Object { $_ }
    
        # get the headers from the first object in the correct order
        Write-Verbose "Retrieving column headers"
        $headers = $Data[0].PSObject.Properties | ForEach-Object {$_.Name }
    
        # calculate the maximum width for each column
        Write-Verbose "Calculating column widths"
        $columnWidths  = @{}
        foreach ($item in $Data) {
            foreach ($column in $headers) {
                $length = [Math]::Max($item.$column.Length, $column.Length)
                if ($column -ne $headers[-1]) { $length += $Gap }
                if ($columnWidths[$column]) { $length = [Math]::Max($columnWidths[$column], $length) }
                $columnWidths[$column] = $length
            }
        }
    
        # output the headers, all left-aligned
        $line = foreach ($column in $headers) {
            "{0, -$($columnWidths[$column])}" -f $column
        }
        # output the first (header) line
        $line -join ''
    
        # regex to test for numeric values
        $reNumeric = '^[+-]?\s*(?:\d{1,3}(?:(,?)\d{3})?(?:\1\d{3})*(\.\d*)?|\.\d+)$'
    
        # next go through all data lines and output formatted rows
        foreach ($item in $Data) {
            $line = foreach ($column in $headers) {
                $padding = $columnWidths[$column]
                # if the value is numeric, align right, otherwise align left
                if ($item.$column -match $reNumeric) { 
                    $padding -= $Gap
                    "{0, $padding}{1}" -f $item.$column, (' ' * $Gap)
                }
                else {
                    "{0, -$padding}" -f $item.$column
                }
            }
            # output the line
            $line -join ''
        }
    }
    

    有了这些功能,使用你的例子,你可以这样做:

    # get the fixed-width data from file and convert it to an array of PSObjects 
    $data = (Get-Content -Path 'D:\Test.txt') | ConvertFrom-FixedWidth -AllowRaggedContent -Verbose
    
    # now you can remove any column like this
    $data = $data | Select-Object * -ExcludeProperty 'AGE'
    
    # show on screen
    $data | Format-Table -AutoSize
    
    # save to disk as new fixed-width file
    $data | ConvertTo-FixedWidth -Gap 3 -Verbose | Set-Content -Path 'D:\Output.txt'
    
    # or you can save it as regular CSV to disk
    $data | Export-Csv -Path 'D:\Output.csv' -NoTypeInformation
    

    屏幕上的结果:

    NAME          READY STATUS  RESTARTS IP        NODE  NOMINATED NODE READINESS GATES
    ----          ----- ------  -------- --        ----  -------------- ---------------
    me-pod-name   2/2   Running 0        10.0.0.10 node1 <none>         <none>         
    me-pod-name-2 1/1   Running 0        10.0.1.20 node2 <none>         <none>         
    me-pod-name-3 1/1   Running 0        10.0.0.30 node3 <none>         <none>
    

    结果保存到固定宽度文件:

    NAME            READY   STATUS    RESTARTS   IP          NODE    NOMINATED NODE   READINESS GATES
    me-pod-name     2/2     Running          0   10.0.0.10   node1   <none>           <none>         
    me-pod-name-2   1/1     Running          0   10.0.1.20   node2   <none>           <none>         
    me-pod-name-3   1/1     Running          0   10.0.0.30   node3   <none>           <none>
    

    【讨论】:

    • 谢谢你。虽然我可能会修改它以更好地匹配某些情况,但它绝对是最适合包罗万象的场景的解决方案。我希望这将有助于其他人轻松找到它们。我使用的完整命令管道如下(我个人不喜欢它通过文件,但为了简单起见):kubectl get pod | ConvertFrom-FixedWidth |Select-Object * -ExcludeProperty 'AGE' | ConvertTo-FixedWidth
    【解决方案2】:

    一个相当晚的答案......
    我有一个与@Theo 类似的解决方案,其形式是一个名为ConvertFrom-SourceTable 的PowerShell cmdlet,我正在尝试维护它。它自动识别许多已知的表格格式并相应地将它们转换为对象,有关详细信息,请参阅相关project site
    我同意@mklement0 的观点,您应该尝试从源头解决问题,但这并不总是可能的。此外,如果有一个固定宽度文本表的标准,它将把语句放在一个完全不同的角度,知道最接近固定宽度文本表的可能是CSV file,它的格式没有完全标准化,只支持字符串(在 PowerShell 中),最重要的是,人类难以阅读。从我的角度来看:如果人类可以阅读它,那么程序应该能够解释它。程序应该帮助人类是可能的,反之亦然。
    与 Theo 的解决方案相反,我的 cmdlet 没有用 2 个或更多空格定义列边界(因为不能保证列只用单个空格分隔),而是结合了标题、标尺和后面的数据.无论如何,这实际上是我正在处理给定表的问题的原因:NOMINATED NODE 列被分成两个单独的列,导致重复的 NODEcolumn。
    考虑到基于 2 个或更多空格来拆分列是不自然的,在此表中很明显 NOMINATED NODEREADINESS GATES 是单列,因为在 NODE - 和 GATES 标题下没有数据文本。
    这个约束让我忙了一段时间,但现在已包含在 ConvertFrom-SourceTable 的最后一次更新中:

    Install-Script -Name ConvertFrom-SourceTable # https://www.powershellgallery.com/packages/ConvertFrom-SourceTable
    . .\ConvertFrom-SourceTable.ps1              # Load (dot-source) the script
    
    $Data = Get-Content .\Table.txt -Raw | ConvertFrom-SourceTable
    

    注意:您也可以考虑省略-Raw 开关和每一行。这将使用更少的内存,但ConvertFrom-SourceTable cmdlet 只能根据第一个数据行而不是整个表来做出决定(例如列识别)。

    # now you can remove any column like as per Theo's answer
    $Data = $Data | Select-Object * -ExcludeProperty 'AGE'
    
    # and create a table without horizontal ruler
    $Header = [Ordered]@{}
    $Data[0].PSObject.Properties | ForEach-Object {$Header[$_.Name] = $_.Name}
    [PSCustomObject]$Header, $Data | Format-Table -HideTableHeaders | Set-Content .\NewTable.txt
    

    结果

    NAME          READY STATUS  RESTARTS IP        NODE  NOMINATED NODE READINESS GATES
    me-pod-name   2/2   Running 0        10.0.0.10 node1 <none>         <none>
    me-pod-name-2 1/1   Running 0        10.0.1.20 node2 <none>         <none>
    me-pod-name-3 1/1   Running 0        10.0.0.30 node3 <none>         <none>
    

    请注意,ConvertFrom-SourceTable cmdlet 仍然能够读回上面的结果表,尽管列由单个空格分隔

    【讨论】:

      【解决方案3】:

      顺便说一句:对于给定的实用程序,您可以 通过using its options to emit a machine-parseable data format而不是漂亮的打印文本(例如JSON)解决的问题,PowerShell可能能够将其解析为对象(例如@ 987654324@),然后您可以轻松地从中选择属性并使用 PowerShell 的 Format-* cmdlet 显示格式。


      在 PowerShell 中解析和转换固定宽度的柱状文本输出:

      通常,最好将输入文本解析为自定义对象([pscustomobject] 实例),这使得它们适合进一步的编程处理以及使用 PowerShell 灵活的重新格式化 Format-* cmdlet。

      在没有预先知识的情况下,这种解析可以在多大程度上自动化取决于给定实用程序的固定宽度输出的具体情况(假设应该修剪每列中的尾随空格):

      • 自动解析

        • 如果满足以下条件,则可以自动确定列和值的范围:

          • 各列由分隔符字符串(至少一个字符)分隔。
          • 并且列名和行值本身没有此分隔符字符串的嵌入实例(尾随空格除外)。
        • 自定义函数Select-Column,其源代码如下所示,可以通过描述分隔符字符串的正则表达式执行此自动解析

      • 否则,您必须通过预先知道的列宽/起始位置进行解析

        • 也就是说,您必须知道所有单个列的宽度/它们的起始位置并以此为基础进行解析。 Theo's answer 展示了如何做到这一点。

      kubectl 的情况下,列由 2 个空格分隔,而列名仅包含 1 个空格,并且值为 none(在您的示例输出中),因此先决条件满足自动解析。

      自定义函数Select-Column,源码如下,可以通过正则表达式  +作为列分隔符表达式(2个或多个空格)将文本解析成自定义对象([pscustomobject]实例),得到的对象为适合在 PowerShell 中进行进一步的编程处理和格式化。

      您可以简单地将kubectl 输出传递给它。 最多有 4 个选定列,生成的对象被隐式格式化为表格(隐含Format-Table);有 5 个或更多,作为列表(隐式 Format-List)。 您可以通过管道显式传递给Format-* cmdlet 以控制格式。

      # Parse all columns except 'Age' into custom objects and format them as a table:
      kubectl ... | Select-Column '  +' -Exclude Age | Format-Table
      

      以上产出:

      NAME          READY STATUS  RESTARTS IP        NODE  NOMINATED NODE READINESS GATES
      ----          ----- ------  -------- --        ----  -------------- ---------------
      me-pod-name   2/2   Running 0        10.0.0.10 node1 <none>         <none>
      me-pod-name-2 1/1   Running 0        10.0.1.20 node2 <none>         <none>
      me-pod-name-3 1/1   Running 0        10.0.0.30 node3 <none>         <none>
      

      您可以将此类调用包装在自定义函数中,包括到Format-Table 的管道,但请注意,执行后者意味着输出再次不适合编程处理,因为Format-* 调用输出格式化指令,不是数据。

      如果您希望输出为表格格式默认情况下,即使对于 5 个或更多属性,也需要做更多工作:您必须为输出对象并为该类型定义formatting data


      Select-Column源码:

      function Select-Column {
        <#
        .SYNOPSIS
        Parses columnar text data into custom objects.
      
        .DESCRIPTION
        Parses line-based columnar text data into custom objects, based on a 
        column-separator pattern specified as a regular expression.
      
        By default, the values from all columns are returned as properties of the output
        objects.
        Use -Name to extract only given columns, or -ExcludeName to exclude columns.
      
        .PARAMETER SeparatorPattern
        A regular expression specifying what text separates the column names / values
        in the input text.
      
        The default is ' +', i.e. any run of one or more spaces, which works with
        fixed-width columns whose column names and values have no embedded spaces.
      
        .PARAMETER Name
        The names of one more columns to extract from the input text.
        These names must match existing columns.
        By default, all columns are returned.
      
        .PARAMETER Exclude
        The names of one more input columns to exclude from the properties of the output
        objects.
        These names must match existing columns.
      
        .PARAMETER InputObject
        This parameter receives the individual lines of input text via the pipeline.
        the pipeline.
      
        Alternatively, you can pass the input text as a single multi-line string, both
        via the pipeline and directly to this parameter.
      
        .EXAMPLE
        'col1 col2', 'val1 val2' | Select-Column
      
        Converts the line-by-line input into custom objects with properties 'col1'
        and 'col2' whose values are 'val1' and 'val2', based on runs of 1 or more
        spaces acting as separators.
      
        .EXAMPLE
        'col 1  col 2', 'val1  val2' | Select-Column ' {2,}' -Exclude 'col 2'
      
        Converts the line-by-line input into custom objects based on two or more spaces
        as separators, excluding values from column 'col 2'.
        #>
      
        [CmdletBinding(DefaultParameterSetName = 'All')]
        param(
          [Parameter(Position = 0)]
          [Alias('s')]
          [string[]] $SeparatorPattern = ' +'
          ,
          [Parameter(ParameterSetName = 'Include', Position = 1)]
          [Alias('n')]
          [string[]] $Name
          ,
          [Parameter(ParameterSetName = 'Exclude')]
          [Alias('x')]
          [string[]] $ExcludeName
          ,
          [Parameter(Mandatory, ValueFromPipeline)]
          [string] $InputObject
        )
      
        begin {
          Set-StrictMode -Version 1
          $lineIndex = 0
        }
      
        process {
          foreach ($line in $InputObject -split '\r?\n' -ne '') {
      
            # Split the line into colum names / fields.
            $fields = $line -split $SeparatorPattern
            # Process the header row
            if ($lineIndex++ -eq 0) {
              $ndx = 0
              # Map column names to their indices.
              $nameToIndex = @{ }
              foreach ($n in $fields) {
                $nameToIndex[$n] = $ndx++
              }
              # Based on the given column names, build a list of indices
              # to extract, and create an ordered hashtable with the specified
              # column names to serve as a template for the output objects.
              $unknownName = $null
              if ($Name) {
                # only the specified columns
                $unknownName = (Compare-Object -PassThru $fields $Name).Where({ $_.SideIndicator -eq '=>' })
              }
              elseif ($ExcludeName) {
                # all but the specified columns
                $Name, $unknownName = (Compare-Object -PassThru $fields $ExcludeName).Where({ $_.SideIndicator -eq '<=' }, 'Split')
              }
              else {
                $Name = $fields # default to all columns
              }
              if ($unknownName) { Throw "Unknown column name(s): $unknownName" }
              if (-not $Name) { Throw "You have selected no output columns." }
              $oht = [ordered] @{ }
              $outColIndices = foreach ($n in $Name) {
                $oht[$n] = $null 
                $nameToIndex[$n] # output the index
              }
            }
            # Process a data row.
            else {
              # Fill in the ordered hashtable with this line's field values...
              $ndx = 0
              foreach ($n in $Name) {
                $oht[$n] = $fields[$outColIndices[$ndx++]]
              }
              # ... and construct and output a custom object from it.
              [pscustomobject] $oht
            }
          }
        }
      }
      

      【讨论】:

      • 我想你在这里错过了我的意图。 Kubectl 只是一个例子。我使用许多这样的工具。过去,我确实手动格式化了原始输出,为命令编写了我赢得的包装器,并单独处理了问题。但是,这不是一个好的解决方案,因为我无法为我用于集群/云管理的每个 CLI 工具编写自己的包装器。他们太多了。并不是每个都会让我有可能弄乱输出格式或轻松地从 JSON 等原始格式重新创建输出。我正在寻找可重复的解决方案。
      • Select-Column 函数确实可以满足我的要求,并且 可以用来代替 Theo 的上述答案。唉,不可能同时选择它们作为有效响应。
      • @user12363468:理解了意图;我已经大幅修改了答案,并附有 Theo's answer for contrast 的链接。重新接受答案:虽然您只能接受一个,但拥有 15 个或更多声望点的您还可以投票其他有用的答案。
      【解决方案4】:

      这里有一个方法。先把所有数据放到一个纯文本文件("C:\Temp.txt") 然后这个脚本会先把所有数据放到一个csv文件中,然后格式化。

      Get-Content "C:\Temp.txt" | Export-CSV "C:\Temp.csv"
      Import-CSV "C:\Temp.csv" | Select-Object NAME, READY, STATUS, RESTARTS, IP, NODE, NOMINATED NODE, READINESS GATES
      Remove-Item "C:\Temp.txt"
      Remove-Item "C:\Temp.csv"
      

      或者,如果任何命令生成了输出,则无需将其放入文本文件中。

      TheCommand | Export-CSV "C:\Temp.csv"
      Import-CSV "C:\Temp.csv" | Select-Object NAME, READY, STATUS, RESTARTS, IP, NODE, NOMINATED NODE, READINESS GATES
      Remove-Item "C:\Temp.txt"
      Remove-Item "C:\Temp.csv"
      

      【讨论】:

      • 虽然它是一种方法,但我想避免它通过文件。更重要的是,我不想为每个可以显示的列制作所需的属性列表。有些表格太动态了。
      猜你喜欢
      • 2012-10-26
      • 1970-01-01
      • 1970-01-01
      • 2022-06-24
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2020-11-17
      相关资源
      最近更新 更多