html2text 命令行 打破 html

Question

我试图弄清楚为什么 html2text 会破坏我的 HTML：

<div><table> <tbody> <tr> <td> <span><strong><a href="/pages/about_paul_221673.cfm"><span>About</span></a></strong></span></td> <td> <span><strong><span>&bull;</span></strong></span></td> <td> <span><strong><a href="/pages/contact_us_222511.cfm"><span>Contact</span></a></strong></span></td> <td> <span><strong><span>&bull;</span></strong></span></td> <td> <span><strong><a><span>Maths Games Order</span></a></strong></span></td> <td> <span><strong><span>&bull;</span></strong></span></td> <td> <span><strong><a href="/pages/faqs_222510.cfm"><span>FAQ</span></a></strong></span></td> </tr> </tbody> </table>s<div> <span><strong>Broadbent Maths Ltd<br> 3 High Street, Welbourn, Lincoln, LN5 0NH </strong></span></div> </div>

处理它：

cat "/home/spider/original-file.txt" | html2text -utf8 -nobs -style pretty

当我运行它时，我得到：

由于输入序列无效，输入重新编码失败。未转换的部分 文字如下。 ▒联系方式▒数学游戏订购▒FAQ

s Broadbent Maths Ltd 3 High Street, Welbourn, Lincoln, LN5 0NH

当我运行 Devel::Peek::Dump() (Perl) 时，我看到的字符串是：

SV = PV(0x564c0a72c860) at 0x564c09967c80
  REFCNT = 1
  FLAGS = (POK,IsCOW,pPOK,UTF8)
  PV = 0x564c0a58bc60 "\n<div><table> <tbody> <tr> <td> <span><strong><a href=\"/pages/about_paul_221673.cfm\"><span>About</span></a></strong></span></td> <td> <span><strong><span>&bull;</span></strong></span></td> <td> <span><strong><a href=\"/pages/contact_us_222511.cfm\"><span>Contact</span></a></strong></span></td> <td> <span><strong><span>&bull;</span></strong></span></td> <td> <span><strong><a><span>Maths Games Order</span></a></strong></span></td> <td> <span><strong><span>&bull;</span></strong></span></td> <td> <span><strong><a href=\"/pages/faqs_222510.cfm\"><span>FAQ</span></a></strong></span></td> </tr> </tbody> </table>s<div> <span><strong>Broadbent Maths Ltd<br> 3 High Street, Welbourn, Lincoln, LN5 0NH </strong></span></div> </div>\n"\0 [UTF8 "\n<div><table> <tbody> <tr> <td> <span><strong><a href="/pages/about_paul_221673.cfm"><span>About</span></a></strong></span></td> <td> <span><strong><span>&bull;</span></strong></span></td> <td> <span><strong><a href="/pages/contact_us_222511.cfm"><span>Contact</span></a></strong></span></td> <td> <span><strong><span>&bull;</span></strong></span></td> <td> <span><strong><a><span>Maths Games Order</span></a></strong></span></td> <td> <span><strong><span>&bull;</span></strong></span></td> <td> <span><strong><a href="/pages/faqs_222510.cfm"><span>FAQ</span></a></strong></span></td> </tr> </tbody> </table>s<div> <span><strong>Broadbent Maths Ltd<br> 3 High Street, Welbourn, Lincoln, LN5 0NH </strong></span></div> </div>\n"]
  CUR = 725
  LEN = 736
  COW_REFCNT = 1

如果我删除第一位：

<div><table>

效果很好！我不明白为什么它会在那里破裂 - 我觉得一切都很好？

Answer 1

好的，我想我已经解决了。在这种情况下，由于某种原因`• 正在破坏它。我用“-”替换了它，它现在可以工作了

html2text -utf8 -nobs -o test-out.txt test.co.uk.txt

html2text 与 HTML 实体打断有点奇怪？

更新：问题原来是当他们将页面作为utf-8 提供元数据时，它是作为iso-8859-1 从服务器传递的。所以我所做的是解析出服务器头并在保存之前进行比较 - 如果它是windows-1252，那么我会使用这个命令而不是解析出来：

html2text -ansi -nobs -o test-out.txt test.co.uk.txt