如何在一组字符串中删除 r 中的这些特殊字符：’s,…

Question

我有这个包含特殊字符的字符串，我无法从主数据框中删除这些字符但是，当我通过 dft 准备一个单独的对象然后我使用以下代码时，我能够删除特殊字符字符。

dft <- "rt shibxwarrior hodl trust processsome great things horizon folks shib shib shiba shibainu shibar… rt askthedr just bought m usd worth shib think it’s robinhoodapp shibaarmy"

rmSpec <- "â|€|¦|â|€™|" # The "|" designates a logical OR in regular expressions.

s.rem <- gsub(rmSpec, "", dft) # gsub replace any matches in remSpec and replace them with "".
s.rem

但是当我在主数据框上使用相同的代码时，如下以不同的行（推文）的形式，相同的代码将不起作用并显示错误：UseMethod中的错误（“检查” ， X） ： 没有适用于“字符”类对象的“检查”方法

[1] rt shibxwarrior hodl 信任过程一些伟大的事情地平线人 shib shib shiba shibainu shibar... [2] rt askthedr 刚买了 m 美元，价值 shib 认为它是 robinhoodapp shibaarmy
[3] rt bitshiba 发送 shib 关注转推推文 uufefufcd
[4] rt shibinform 想要 shib 列出 robinhoodappuf 是 是 是 ubufef ubufef ubufef
[5] rt shiblucky shib 赠品只需转发关注

请您帮忙，谢谢。

Answer 1

仅提取我们可能使用的字母和数字，

library(stringr)
    
dft <- "rt shibxwarrior hodl trust processsome great things horizon folks shib shib shiba shibainu shibar… rt askthedr just bought m usd worth shib think it’s robinhoodapp shibaarmy"

str_replace_all(dft, "[^a-zA-Z0-9]", " ")
[1] "rt shibxwarrior hodl trust processsome great things horizon folks shib shib shiba shibainu shibar    rt askthedr just bought m usd worth shib think it   s robinhoodapp shibaarmy"

Answer 2

如前所述，首选方法是使用正确的编码读入您的数据，但有时您的数据只是损坏了。我的 FixEncoding 函数创建了一个名为 vector 的查找来解决这个问题（我过去在处理错误存储的旧 csv 文件时遇到的最多 3 个错误编码。您可以使用所有 unicode 字符并错误编码。这样您可以将错误翻译回来.

FixEncoding <- function() {
  # create the unicode ranges from https://www.i18nqa.com/debug/utf8-debug.html
  range <- c(sprintf("%x", seq(strtoi("0xa0"), strtoi("0xff"))))
  unicode <- vapply(range, FUN.VALUE = character(1), function(x) { parse(text = paste0("'\\u00", x, "'"))[[1]] })
  # add the ones that are missing (red ones in https://www.i18nqa.com/debug/utf8-debug.html)
  unicode <- c(c("\u0168", "\u0152", "\u017d", "\u0153", "\u017e", "\u0178", "\u2019", "\u20ac", "\u201a", "\u0192", "\u201e", "\u2026", "\u2020", "\u2021", "\u02c6", "\u2030", "\u0160", "\u2030"), unicode)
  once <- vapply(unicode, FUN.VALUE = character(1), function(x) { 
    Encoding(x) <- "Windows-1252"
    iconv(x, to = "UTF-8")
  })
  fix_once <- unicode
  names(fix_once) <- once
  twice <- vapply(once, FUN.VALUE = character(1), function(x) { 
    Encoding(x) <- "Windows-1252"
    iconv(x, to = "UTF-8")
  })
  fix_twice <- unicode
  names(fix_twice) <- twice
  triple <- vapply(twice, FUN.VALUE = character(1), function(x) { 
    Encoding(x) <- "Windows-1252"
    iconv(x, to = "UTF-8")
  })
  fix_triple <- unicode
  names(fix_triple) <- triple
  fixes <- c(fix_triple, fix_twice, fix_once)
  return(fixes)
}

fixes <- FixEncoding()

str(fixes)

 Named chr [1:342] "U" "Œ" "Ž" "œ" "ž" "Ÿ" "’" "\200" "‚" "ƒ" "„" "…" "†" "‡" "\210" "‰" "Š" "‰" " " "¡" "¢" "£" "¤" "¥" "¦" "§" "¨" "©" "ª" "«" "¬" "­" "®" "¯" "°" "±" "²" "³" "´" ...
 - attr(*, "names")= chr [1:342] "Ãâ\200¦Ã‚¨" "Ãâ\200¦Ã¢â‚¬â„¢" "Ãâ\200¦Ã‚½" "Ãâ\200¦Ã¢â‚¬Å“" ...

dft <- "rt shibxwarrior hodl trust processsome great things horizon folks shib shib shiba shibainu shibar… rt askthedr just bought m usd worth shib think it’s robinhoodapp shibaarmy"

stri_replace_all_fixed(dft, names(fixes), fixes, vectorize_all = F)

[1] "rt shibxwarrior hodl trust processsome great things horizon folks shib shib shiba shibainu shibar… rt askthedr just bought m usd worth shib think it’s robinhoodapp shibaarmy"

发生情况的简单示例

让lélèlölã 使用 unicode "\u006C\u00E9\u006C\u00E8\u006C\u00F6\u006C\u00E3"

messy <- "\u006C\u00E9\u006C\u00E8\u006C\u00F6\u006C\u00E3"

messy
# [1] "lélèlölã"

# this is just what happens if files get corrupt by saving in wrong encodings, we did that unicode character by unicode character in my fix function.
Encoding(messy) <- "Windows-1252"
messy <- iconv(messy, to = "UTF-8")

messy
# [1] "lélèlölã" # once badly encoded
# [1] "lélèlölã" # twice badly encoded
# [1] "lélèlölã" # three times!

# All three strings above would be fixed
stri_replace_all_fixed(messy, names(fixes), fixes, vectorize_all = F)
# [1] "lélèlölã"

# other simple replacements as suggested would give us `llll` or `lÃlÃlÃlÃ`?