如何在 R 中阅读带有印地语文本的 PDF？

Question

我有多个带有印地语文本的打字 PDF。我需要使用所有 PDF 创建一个数据框以进行进一步分析。我为此使用 R。

> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Mojave 10.14.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] pdftools_3.0.1  textreadr_1.0.2 tm_0.7-8        NLP_0.2-1      

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7        pillar_1.6.1      compiler_4.0.3    tools_4.0.3       lfe_2.8-6         lifecycle_1.0.0  
 [7] tibble_3.1.2      lattice_0.20-41   pkgconfig_2.0.3   rlang_0.4.11      Matrix_1.2-18     DBI_1.1.1        
[13] parallel_4.0.3    xfun_0.23         dplyr_1.0.6       xml2_1.3.2        generics_0.1.0    vctrs_0.3.8      
[19] askpass_1.1       grid_4.0.3        tidyselect_1.1.0  data.table_1.13.2 glue_1.4.2        qpdf_1.1         
[25] R6_2.5.0          fansi_0.4.1       sp_1.4-4          Formula_1.2-4     purrr_0.3.4       magrittr_2.0.1   
[31] ellipsis_0.3.2    assertthat_0.2.1  xtable_1.8-4      sandwich_3.0-1    utf8_1.1.4        tinytex_0.32     
[37] slam_0.1-48       crayon_1.4.1      zoo_1.8-9    

我尝试过使用以下内容：

library(tm)
library(pdftools)

file <- 'pdf_file.pdf'

dat = readPDF(control=list(text="-layout"))(elem=list(uri=file), 
                                            language="UTF-8", id="id1") 

library(textreadr)

pdf_dat <- read_pdf(system.file("pdf_file.pdf", package = "textreadr"))

pdf_text(file)

但是，我得到的结果看起来像：

   27 ªF³F½FSXeX XY2012\n\n                                                                                                                                                                                       ¸FûQe ³FZ d½FIYFÀF °Fû dIY¹FF W` : IYFa¦FiZÀF\n                                                                                                                                                                                         w ¦F„þSF°F IYFa¦FiZÀF IZY d½FÄFF´F³F ¸FZÔ dQJe          ¦F„þSF°Fe IYû d½FIYFÀF ´F„÷Y¿F ¶F°FF°FZ W„E SFª¹F IZY        IYFa¦FiÀZ F ³FZ ¹FWX ÀFaQVZ F QZ³FZ IYe IYFZdVFVF þøYSX            QþʳF C°ÀF½FûÔ IYF ¶FûÓF OXF»F³FZ IYF AFSû´F\n                                                                                                                                                                                                                                                                                                                                                                                                                                                    ³F¦FSX ÀFaÀIYSX¯F\n   

可从here 下载用于此目的的示例 PDF。

谢谢！

Answer 1

根据要求，我在保留 Encoding 时用于阅读 pdf 的代码。

与 tm：

library(tm)
library(pdftools)

text <- pdf_text("NBT-16-Aug-2021.pdf")
corp <- VCorpus(VectorSource(text))
dtm <- DocumentTermMatrix(corp)
inspect(dtm[1:3, 1:3])
<<DocumentTermMatrix (documents: 3, terms: 3)>>
Non-/sparse entries: 2/7
Sparsity           : 78%
Maximal term length: 5
Weighting          : term frequency (tf)
Sample             :
    Terms
Docs                       \033पेज                             'अपनी                      'अपर
   1                            5                                 1                         0
   2                            0                                 0                         0
   3                            0                                 0                         0

或

dat <- VCorpus(URISource("NBT-16-Aug-2021.pdf", mode = ""),
               readerControl = list(reader = readPDF(engine = "pdftools")))

dtm <- DocumentTermMatrix(dat)
inspect(dtm)
<<DocumentTermMatrix (documents: 1, terms: 3)>>
Non-/sparse entries: 3/0
Sparsity           : 0%
Maximal term length: 5
Weighting          : term frequency (tf)
Sample             :
                     Terms
Docs                                        \033पेज                             'अपनी                      'अपर
  NBT-16-Aug-2021.pdf                            5                                 1                         1

使用量子：

library(readtext)
library(quanteda)
text <- readtext("NBT-16-Aug-2021.pdf")
corp <- corpus(text)
my_dfm <- dfm(tokens(corp))
head(featfreq(my_dfm), 20)

                             yo                                        u 
                               2                                        3 
                               n                                        g 
                              14                                        3 
                               i                                        d 
                               2                                        1 
                               a                                        > 
                               3                                        5 
                              pa                                        p 
                               1                                        1 
                               e                                        r 
                               2                                        2 
                               ि                                    पर्यंका 
                               3                                       10 
                               ने                                     भारत 
                             718                                       74 
                              की                                      मदद 
                             915                                       19 
                               क                                        ि 
                            1870                                     2567 

我更喜欢使用 quanteda，因为它可以在所有操作中更好地保​​持 UTF-8。当您尝试清洁时，tm 有时会丢失它们。 quanteda 也可以开箱即用地并行工作。