从第三次观察开始，在 R 中拟合 Plotly 的黄土曲线

Question

我正在使用 LOESS Smoother 在 Scatterplot 上关注 this tutorial，但我希望能够将二阶导数应用于 LOESS 平滑线以检查它达到最大值的位置，这样我就可以知道有多少集群是最佳的，就好像它是k-means 的肘部。

perplexi <- structure(list(Perplexity = c(NA, NA, 660, 596, 552, 480, 464, 
                      415, 399, 370, 349, 340, 327, 314, 288), Clusters = c(1, 2, 3, 
                      4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)), class = "data.frame", row.names = c(NA, 
                      -15L))

library(plotly)

p <- plot_ly(perplexi[3:15,],
             x = ~Clusters,
             color = I("black")) %>% 
  add_markers(y = ~Perplexity) %>% 
  add_lines(y = ~fitted(loess(Perplexity ~ Clusters)),
                         line = list(color = 'lightblue'),
                         name = "Loess Smoother",
                         showlegend = F) %>% 
  layout(xaxis = list(title = 'Clusters'),
         yaxis = list(title = 'Perplexity')) %>% 
  add_trace(y = ~Perplexity,
            name = 'Perplexity',
            mode = 'markers',
            showlegend = F)

p

d1 <- diff(perplex); k <- which.max(abs(diff(d1) / diff(perplex[-1])))

有人可以指出下一步该怎么做吗？我希望 k 用于平滑线而不是实际数字，因此我知道要执行多少个主题。

Answer 1

一种方法是在 plotly 之外拟合黄土，然后求导数。

loess.result <-loess.smooth(perplexi$Clusters, y=perplexi$Perplexity, evaluation = 20)
slopes <- diff(loess.result$x)/diff(loess.result$y)

plot_ly(perplexi[3:15,],
             x = ~Clusters,
             color = I("black")) %>% 
  add_markers(y = ~Perplexity) %>% 
  add_lines(y = ~fitted(loess(Perplexity ~ Clusters)),
                         line = list(color = 'lightblue'),
                         name = "Loess Smoother") %>% 
  layout(xaxis = list(title = 'Clusters'),
         yaxis = list(title = 'Perplexity')) %>% 
  add_trace(y = ~Perplexity,
            name = 'Perplexity',
            mode = 'markers',
            showlegend = F) %>%
  add_trace(x = loess.result$x[-1], y = slopes * -10000, mode = "line", name = "Loess First Derivative")