【问题标题】:How can I extract subscript / superscript properly from a PDF using iTextSharp?如何使用 iTextSharp 从 PDF 中正确提取下标/上标?
【发布时间】:2016-02-03 05:34:07
【问题描述】:

iTextSharp 可以很好地从 PDF 文档中提取纯文本,但我在处理技术文档中常见的下标/上标文本时遇到了问题。

TextChunk.SameLine() 要求两个块具有相同 垂直定位“在”同一行,上标或下标文本不是这种情况。例如,在本文档第 11 页的“燃烧效率”下:

http://www.mass.gov/courts/docs/lawlib/300-399cmr/310cmr7.pdf

预期文本:

monoxide (CO) in flue gas in accordance with the following formula: C.E. = [CO2 /(CO + CO2)]

结果文本:

monoxide (CO) in flue gas in accordance with the following formula: C.E. = [CO /(CO + CO )] 
2 2 

我将 SameLine() 移动到 LocationTextExtractionStrategy 并为其读取的私有 TextChunk 属性创建了公共 getter。这使我可以在我自己的子类中动态调整容差,如下所示:

public class SubSuperStrategy : LocationTextExtractionStrategy {
  public int SameLineOrientationTolerance { get; set; }
  public int SameLineDistanceTolerance { get; set; }

  public override bool SameLine(TextChunk chunk1, TextChunk chunk2) {
    var orientationDelta = Math.Abs(chunk1.OrientationMagnitude
       - chunk2.OrientationMagnitude);
    if(orientationDelta > SameLineOrientationTolerance) return false;
    var distDelta = Math.Abs(chunk1.DistPerpendicular
       - chunk2.DistPerpendicular);
    return (distDelta <= SameLineDistanceTolerance);
    }
}

使用3 中的SameLineDistanceTolerance,这可以更正子/超级块被分配到的,但是文本的相对位置偏离了:

monoxide (CO) in flue gas in accordance with the following formula:   C.E. = [CO /(CO + CO )] 2 2

有时块会插入文本中间的某个位置,有时(如本例所示)在末尾。无论哪种方式,它们最终都不会出现在正确的位置。我怀疑这可能与字体大小有关,但我无法理解这段代码的内容。

有没有人找到另一种方法来处理这个问题?

(如果有帮助,我很乐意提交包含我的更改的拉取请求。)

【问题讨论】:

  • 您可能想看看 textRenderInfo 中的 getRise,它经常用于下标和上标,但并不总是使用。 api.itextpdf.com/itext/com/itextpdf/text/pdf/parser/…
  • @KimvdLinde 不幸的是它在这里没有帮助:12 0 0 12 91.5 443.04 Tm [(monox)-12(ide \(CO\) in flue)10( g)10(as in )-9(acc)12(ordanc)11(e with )-18(the foll)-9(owing formula: )-16(C.E. )-12(= [)-13(CO )-131(/\(CO + CO )-127(\)] )]TJ EMC /P &lt;&lt;/MCID 22 &gt;&gt;BDC 0 Tw 6.96 0 0 6.96 473.28 440.4 Tm (2)Tj 7.914 0 Td (2 )Tj - 下标确实是在行的其余部分之后绘制的,并且位置和大小通过完整的文本矩阵更改来更改。因此,确实需要一种对文本行的概念有更好理解的排序文本提取策略。
  • 你是对的,这需要确定起点和终点,并使用它来对它们进行适当的排序。

标签: c# pdf itextsharp itext


【解决方案1】:

要正确提取这些下标和上标,需要一种不同的方法来检查两个文本块是否在同一行上。以下类代表了一种这样的方法。

我更熟悉 Java/iText;因此,我首先在 Java 中实现了这种方法,然后才将其翻译为 C#/iTextSharp。

使用 Java 和 iText 的方法

我正在使用当前的开发分支 iText 5.5.8-SNAPSHOT。

一种识别线的方法

假设文本行是水平的,并且不同行上字形的边界框的垂直延伸不重叠,可以尝试使用RenderListener 来识别行,如下所示:

public class TextLineFinder implements RenderListener
{
    @Override
    public void beginTextBlock() { }
    @Override
    public void endTextBlock() { }
    @Override
    public void renderImage(ImageRenderInfo renderInfo) { }

    /*
     * @see RenderListener#renderText(TextRenderInfo)
     */
    @Override
    public void renderText(TextRenderInfo renderInfo)
    {
        LineSegment ascentLine = renderInfo.getAscentLine();
        LineSegment descentLine = renderInfo.getDescentLine();
        float[] yCoords = new float[]{
                ascentLine.getStartPoint().get(Vector.I2),
                ascentLine.getEndPoint().get(Vector.I2),
                descentLine.getStartPoint().get(Vector.I2),
                descentLine.getEndPoint().get(Vector.I2)
        };
        Arrays.sort(yCoords);
        addVerticalUseSection(yCoords[0], yCoords[3]);
    }

    /**
     * This method marks the given interval as used.
     */
    void addVerticalUseSection(float from, float to)
    {
        if (to < from)
        {
            float temp = to;
            to = from;
            from = temp;
        }

        int i=0, j=0;
        for (; i<verticalFlips.size(); i++)
        {
            float flip = verticalFlips.get(i);
            if (flip < from)
                continue;

            for (j=i; j<verticalFlips.size(); j++)
            {
                flip = verticalFlips.get(j);
                if (flip < to)
                    continue;
                break;
            }
            break;
        }
        boolean fromOutsideInterval = i%2==0;
        boolean toOutsideInterval = j%2==0;

        while (j-- > i)
            verticalFlips.remove(j);
        if (toOutsideInterval)
            verticalFlips.add(i, to);
        if (fromOutsideInterval)
            verticalFlips.add(i, from);
    }

    final List<Float> verticalFlips = new ArrayList<Float>();
}

(TextLineFinder.java)

RenderListener 尝试通过将文本边界框投影到 y 轴来识别水平文本行。它假设这些投影对于来自不同行的文本不会重叠,即使在下标和上标的情况下也是如此。

这个类本质上是this answer中使用的PageVerticalAnalyzer的简化形式。

按这些行对文本块进行排序

识别出上述行后,可以调整 iText 的 LocationTextExtractionStrategy 以像这样对这些行进行排序:

public class HorizontalTextExtractionStrategy extends LocationTextExtractionStrategy
{
    public class HorizontalTextChunk extends TextChunk
    {
        public HorizontalTextChunk(String string, Vector startLocation, Vector endLocation, float charSpaceWidth)
        {
            super(string, startLocation, endLocation, charSpaceWidth);
        }

        @Override
        public int compareTo(TextChunk rhs)
        {
            if (rhs instanceof HorizontalTextChunk)
            {
                HorizontalTextChunk horRhs = (HorizontalTextChunk) rhs;
                int rslt = Integer.compare(getLineNumber(), horRhs.getLineNumber());
                if (rslt != 0) return rslt;
                return Float.compare(getStartLocation().get(Vector.I1), rhs.getStartLocation().get(Vector.I1));
            }
            else
                return super.compareTo(rhs);
        }

        @Override
        public boolean sameLine(TextChunk as)
        {
            if (as instanceof HorizontalTextChunk)
            {
                HorizontalTextChunk horAs = (HorizontalTextChunk) as;
                return getLineNumber() == horAs.getLineNumber();
            }
            else
                return super.sameLine(as);
        }

        public int getLineNumber()
        {
            Vector startLocation = getStartLocation();
            float y = startLocation.get(Vector.I2);
            List<Float> flips = textLineFinder.verticalFlips;
            if (flips == null || flips.isEmpty())
                return 0;
            if (y < flips.get(0))
                return flips.size() / 2 + 1;
            for (int i = 1; i < flips.size(); i+=2)
            {
                if (y < flips.get(i))
                {
                    return (1 + flips.size() - i) / 2;
                }
            }
            return 0;
        }
    }

    @Override
    public void renderText(TextRenderInfo renderInfo)
    {
        textLineFinder.renderText(renderInfo);

        LineSegment segment = renderInfo.getBaseline();
        if (renderInfo.getRise() != 0){ // remove the rise from the baseline - we do this because the text from a super/subscript render operations should probably be considered as part of the baseline of the text the super/sub is relative to 
            Matrix riseOffsetTransform = new Matrix(0, -renderInfo.getRise());
            segment = segment.transformBy(riseOffsetTransform);
        }
        TextChunk location = new HorizontalTextChunk(renderInfo.getText(), segment.getStartPoint(), segment.getEndPoint(), renderInfo.getSingleSpaceWidth());
        getLocationalResult().add(location);        
    }

    public HorizontalTextExtractionStrategy() throws NoSuchFieldException, SecurityException
    {
        locationalResultField = LocationTextExtractionStrategy.class.getDeclaredField("locationalResult");
        locationalResultField.setAccessible(true);

        textLineFinder = new TextLineFinder();
    }

    @SuppressWarnings("unchecked")
    List<TextChunk> getLocationalResult()
    {
        try
        {
            return (List<TextChunk>) locationalResultField.get(this);
        }
        catch (IllegalArgumentException | IllegalAccessException e)
        {
            e.printStackTrace();
            throw new RuntimeException(e);
        }
    }

    final Field locationalResultField;
    final TextLineFinder textLineFinder;
}

(HorizontalTextExtractionStrategy.java)

这个TextExtractionStrategy 使用TextLineFinder 来识别水平文本行,然后使用这些信息对文本块进行排序。

注意,此代码使用反射来访问私有父类成员。并非所有环境都允许这样做。在这种情况下,只需复制LocationTextExtractionStrategy并直接插入代码即可。

提取文本

现在可以使用这种文本提取策略来提取具有内联上标和下标的文本,如下所示:

String extract(PdfReader reader, int pageNo) throws IOException, NoSuchFieldException, SecurityException
{
    return PdfTextExtractor.getTextFromPage(reader, pageNo, new HorizontalTextExtractionStrategy());
}

(来自ExtractSuperAndSubInLine.java

OP 文档第 11 页“燃烧效率”下的示例文本现在提取如下:

monoxide (CO) in flue gas in accordance with the following formula:   C.E. = [CO 2/(CO + CO 2 )] 

使用 C# 和 iTextSharp 的相同方法

以 Java 为中心的部分的解释、警告和示例结果仍然适用,代码如下:

我正在使用 iTextSharp 5.5.7。

一种识别线的方法

public class TextLineFinder : IRenderListener
{
    public void BeginTextBlock() { }
    public void EndTextBlock() { }
    public void RenderImage(ImageRenderInfo renderInfo) { }

    public void RenderText(TextRenderInfo renderInfo)
    {
        LineSegment ascentLine = renderInfo.GetAscentLine();
        LineSegment descentLine = renderInfo.GetDescentLine();
        float[] yCoords = new float[]{
            ascentLine.GetStartPoint()[Vector.I2],
            ascentLine.GetEndPoint()[Vector.I2],
            descentLine.GetStartPoint()[Vector.I2],
            descentLine.GetEndPoint()[Vector.I2]
        };
        Array.Sort(yCoords);
        addVerticalUseSection(yCoords[0], yCoords[3]);
    }

    void addVerticalUseSection(float from, float to)
    {
        if (to < from)
        {
            float temp = to;
            to = from;
            from = temp;
        }

        int i=0, j=0;
        for (; i<verticalFlips.Count; i++)
        {
            float flip = verticalFlips[i];
            if (flip < from)
                continue;

            for (j=i; j<verticalFlips.Count; j++)
            {
                flip = verticalFlips[j];
                if (flip < to)
                    continue;
                break;
            }
            break;
        }
        bool fromOutsideInterval = i%2==0;
        bool toOutsideInterval = j%2==0;

        while (j-- > i)
            verticalFlips.RemoveAt(j);
        if (toOutsideInterval)
            verticalFlips.Insert(i, to);
        if (fromOutsideInterval)
            verticalFlips.Insert(i, from);
    }

    public List<float> verticalFlips = new List<float>();
}

按这些行对文本块进行排序

public class HorizontalTextExtractionStrategy : LocationTextExtractionStrategy
{
    public class HorizontalTextChunk : TextChunk
    {
        public HorizontalTextChunk(String stringValue, Vector startLocation, Vector endLocation, float charSpaceWidth, TextLineFinder textLineFinder)
            : base(stringValue, startLocation, endLocation, charSpaceWidth)
        {
            this.textLineFinder = textLineFinder;
        }

        override public int CompareTo(TextChunk rhs)
        {
            if (rhs is HorizontalTextChunk)
            {
                HorizontalTextChunk horRhs = (HorizontalTextChunk) rhs;
                int rslt = CompareInts(getLineNumber(), horRhs.getLineNumber());
                if (rslt != 0) return rslt;
                return CompareFloats(StartLocation[Vector.I1], rhs.StartLocation[Vector.I1]);
            }
            else
                return base.CompareTo(rhs);
        }

        public override bool SameLine(TextChunk a)
        {
            if (a is HorizontalTextChunk)
            {
                HorizontalTextChunk horAs = (HorizontalTextChunk) a;
                return getLineNumber() == horAs.getLineNumber();
            }
            else
                return base.SameLine(a);
        }

        public int getLineNumber()
        {
            Vector startLocation = StartLocation;
            float y = startLocation[Vector.I2];
            List<float> flips = textLineFinder.verticalFlips;
            if (flips == null || flips.Count == 0)
                return 0;
            if (y < flips[0])
                return flips.Count / 2 + 1;
            for (int i = 1; i < flips.Count; i+=2)
            {
                if (y < flips[i])
                {
                    return (1 + flips.Count - i) / 2;
                }
            }
            return 0;
        }

        private static int CompareInts(int int1, int int2){
            return int1 == int2 ? 0 : int1 < int2 ? -1 : 1;
        }

        private static int CompareFloats(float float1, float float2)
        {
            return float1 == float2 ? 0 : float1 < float2 ? -1 : 1;
        }

        TextLineFinder textLineFinder;
    }

    public override void RenderText(TextRenderInfo renderInfo)
    {
        textLineFinder.RenderText(renderInfo);

        LineSegment segment = renderInfo.GetBaseline();
        if (renderInfo.GetRise() != 0){ // remove the rise from the baseline - we do this because the text from a super/subscript render operations should probably be considered as part of the baseline of the text the super/sub is relative to 
            Matrix riseOffsetTransform = new Matrix(0, -renderInfo.GetRise());
            segment = segment.TransformBy(riseOffsetTransform);
        }
        TextChunk location = new HorizontalTextChunk(renderInfo.GetText(), segment.GetStartPoint(), segment.GetEndPoint(), renderInfo.GetSingleSpaceWidth(), textLineFinder);
        getLocationalResult().Add(location);        
    }

    public HorizontalTextExtractionStrategy()
    {
        locationalResultField = typeof(LocationTextExtractionStrategy).GetField("locationalResult", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance);
        textLineFinder = new TextLineFinder();
    }

    List<TextChunk> getLocationalResult()
    {
        return (List<TextChunk>) locationalResultField.GetValue(this);
    }

    System.Reflection.FieldInfo locationalResultField;
    TextLineFinder textLineFinder;
}

提取文本

    string extract(PdfReader reader, int pageNo)
    {
        return PdfTextExtractor.GetTextFromPage(reader, pageNo, new HorizontalTextExtractionStrategy());
    }

更新:LocationTextExtractionStrategy 的变化

在iText 5.5.9 - 快照提交53526E4854FCB80C86CBC2E113F7A07401DC9A67(“Refactor LocationTextExtractionStrationStrategry ...”)到1AB350Beae148Be2a4bef5e663b3d67a004ff9f8(“制作textchunklocation a比较 class ...”)LocationTextExtractionStrategy architecture已更改为允许自定义这无需反思。

不幸的是,此更改破坏了上面介绍的 Horizo​​ntalTextExtractionStrategy。对于这些提交之后的 iText 版本,可以使用以下策略:

public class HorizontalTextExtractionStrategy2 extends LocationTextExtractionStrategy
{
    public static class HorizontalTextChunkLocationStrategy implements TextChunkLocationStrategy
    {
        public HorizontalTextChunkLocationStrategy(TextLineFinder textLineFinder)
        {
            this.textLineFinder = textLineFinder;
        }

        @Override
        public TextChunkLocation createLocation(TextRenderInfo renderInfo, LineSegment baseline)
        {
            return new HorizontalTextChunkLocation(baseline.getStartPoint(), baseline.getEndPoint(), renderInfo.getSingleSpaceWidth());
        }

        final TextLineFinder textLineFinder;

        public class HorizontalTextChunkLocation implements TextChunkLocation
        {
            /** the starting location of the chunk */
            private final Vector startLocation;
            /** the ending location of the chunk */
            private final Vector endLocation;
            /** unit vector in the orientation of the chunk */
            private final Vector orientationVector;
            /** the orientation as a scalar for quick sorting */
            private final int orientationMagnitude;
            /** perpendicular distance to the orientation unit vector (i.e. the Y position in an unrotated coordinate system)
             * we round to the nearest integer to handle the fuzziness of comparing floats */
            private final int distPerpendicular;
            /** distance of the start of the chunk parallel to the orientation unit vector (i.e. the X position in an unrotated coordinate system) */
            private final float distParallelStart;
            /** distance of the end of the chunk parallel to the orientation unit vector (i.e. the X position in an unrotated coordinate system) */
            private final float distParallelEnd;
            /** the width of a single space character in the font of the chunk */
            private final float charSpaceWidth;

            public HorizontalTextChunkLocation(Vector startLocation, Vector endLocation, float charSpaceWidth)
            {
                this.startLocation = startLocation;
                this.endLocation = endLocation;
                this.charSpaceWidth = charSpaceWidth;

                Vector oVector = endLocation.subtract(startLocation);
                if (oVector.length() == 0)
                {
                    oVector = new Vector(1, 0, 0);
                }
                orientationVector = oVector.normalize();
                orientationMagnitude = (int)(Math.atan2(orientationVector.get(Vector.I2), orientationVector.get(Vector.I1))*1000);

                // see http://mathworld.wolfram.com/Point-LineDistance2-Dimensional.html
                // the two vectors we are crossing are in the same plane, so the result will be purely
                // in the z-axis (out of plane) direction, so we just take the I3 component of the result
                Vector origin = new Vector(0,0,1);
                distPerpendicular = (int)(startLocation.subtract(origin)).cross(orientationVector).get(Vector.I3);

                distParallelStart = orientationVector.dot(startLocation);
                distParallelEnd = orientationVector.dot(endLocation);
            }

            public int orientationMagnitude()   {   return orientationMagnitude;    }
            public int distPerpendicular()      {   return distPerpendicular;       }
            public float distParallelStart()    {   return distParallelStart;       }
            public float distParallelEnd()      {   return distParallelEnd;         }
            public Vector getStartLocation()    {   return startLocation;           }
            public Vector getEndLocation()      {   return endLocation;             }
            public float getCharSpaceWidth()    {   return charSpaceWidth;          }

            /**
             * @param as the location to compare to
             * @return true is this location is on the the same line as the other
             */
            public boolean sameLine(TextChunkLocation as)
            {
                if (as instanceof HorizontalTextChunkLocation)
                {
                    HorizontalTextChunkLocation horAs = (HorizontalTextChunkLocation) as;
                    return getLineNumber() == horAs.getLineNumber();
                }
                else
                    return orientationMagnitude() == as.orientationMagnitude() && distPerpendicular() == as.distPerpendicular();
            }

            /**
             * Computes the distance between the end of 'other' and the beginning of this chunk
             * in the direction of this chunk's orientation vector.  Note that it's a bad idea
             * to call this for chunks that aren't on the same line and orientation, but we don't
             * explicitly check for that condition for performance reasons.
             * @param other
             * @return the number of spaces between the end of 'other' and the beginning of this chunk
             */
            public float distanceFromEndOf(TextChunkLocation other)
            {
                float distance = distParallelStart() - other.distParallelEnd();
                return distance;
            }

            public boolean isAtWordBoundary(TextChunkLocation previous)
            {
                /**
                 * Here we handle a very specific case which in PDF may look like:
                 * -.232 Tc [( P)-226.2(r)-231.8(e)-230.8(f)-238(a)-238.9(c)-228.9(e)]TJ
                 * The font's charSpace width is 0.232 and it's compensated with charSpacing of 0.232.
                 * And a resultant TextChunk.charSpaceWidth comes to TextChunk constructor as 0.
                 * In this case every chunk is considered as a word boundary and space is added.
                 * We should consider charSpaceWidth equal (or close) to zero as a no-space.
                 */
                if (getCharSpaceWidth() < 0.1f)
                    return false;

                float dist = distanceFromEndOf(previous);

                return dist < -getCharSpaceWidth() || dist > getCharSpaceWidth()/2.0f;
            }

            public int getLineNumber()
            {
                Vector startLocation = getStartLocation();
                float y = startLocation.get(Vector.I2);
                List<Float> flips = textLineFinder.verticalFlips;
                if (flips == null || flips.isEmpty())
                    return 0;
                if (y < flips.get(0))
                    return flips.size() / 2 + 1;
                for (int i = 1; i < flips.size(); i+=2)
                {
                    if (y < flips.get(i))
                    {
                        return (1 + flips.size() - i) / 2;
                    }
                }
                return 0;
            }

            @Override
            public int compareTo(TextChunkLocation rhs)
            {
                if (rhs instanceof HorizontalTextChunkLocation)
                {
                    HorizontalTextChunkLocation horRhs = (HorizontalTextChunkLocation) rhs;
                    int rslt = Integer.compare(getLineNumber(), horRhs.getLineNumber());
                    if (rslt != 0) return rslt;
                    return Float.compare(getStartLocation().get(Vector.I1), rhs.getStartLocation().get(Vector.I1));
                }
                else
                {
                    int rslt;
                    rslt = Integer.compare(orientationMagnitude(), rhs.orientationMagnitude());
                    if (rslt != 0) return rslt;

                    rslt = Integer.compare(distPerpendicular(), rhs.distPerpendicular());
                    if (rslt != 0) return rslt;

                    return Float.compare(distParallelStart(), rhs.distParallelStart());
                }
            }
        }
    }

    @Override
    public void renderText(TextRenderInfo renderInfo)
    {
        textLineFinder.renderText(renderInfo);
        super.renderText(renderInfo);
    }

    public HorizontalTextExtractionStrategy2() throws NoSuchFieldException, SecurityException
    {
        this(new TextLineFinder());
    }

    public HorizontalTextExtractionStrategy2(TextLineFinder textLineFinder) throws NoSuchFieldException, SecurityException
    {
        super(new HorizontalTextChunkLocationStrategy(textLineFinder));

        this.textLineFinder = textLineFinder;
    }

    final TextLineFinder textLineFinder;
}

(HorizontalTextExtractionStrategy2.java)

【讨论】:

  • 抱歉尚未接受。它看起来正是我正在寻找的解决方案,我只是太专注于其他任务而无法返回并测试它......会尽快完成。
【解决方案2】:

我刚刚解决了一个类似的问题,请参阅my question。我将下标检测为在前面文本的升序和降序行之间具有基线的文本。这段代码可能有用:

        Vector thisFacade = this.ascentLine.GetStartPoint().Subtract(this.descentLine.GetStartPoint());
        Vector infoFacade = renderInfo.GetAscentLine().GetStartPoint().Subtract(renderInfo.GetDescentLine().GetStartPoint());
        if (baseVector.Cross(ascent2base).Dot(baseVector.Cross(descent2base)) < 0
            && infoFacade.LengthSquared < thisFacade.LengthSquared - sameHeightThreshols)

圣诞节后的更多细节。

【讨论】:

    猜你喜欢
    • 2012-08-28
    • 2011-08-22
    • 2011-06-02
    • 2017-03-30
    • 1970-01-01
    • 2014-06-06
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多