【问题标题】:Getting unexpected sequence realization in Clojure在 Clojure 中获得意想不到的序列实现
【发布时间】:2021-02-03 23:15:56
【问题描述】:

在尝试对复杂计算进行故障排除时发现了一些奇怪的东西。我遇到了一个奇怪的错误,所以我开始逐步建立计算,并且很早就发现意外实现了一个非常长的序列。我有一个这样的列表组合序列:

(0 1 2 3)
(0 1 2 4)
(0 1 2 5)

for n-choose-k for n = 143 和 k = 4 有点不经意地命名为“combos”。我计划将其作为 seq 进行操作以保持合理的内存消耗,但这失败了:

(def combos (combinations 4 143))
(def semantically-also-combos
  (filter nil? (map identity combos)))

;; this prints instantly and uses almost no memory, as expected
(println (first combos)) ; prints (0 1 2 3)

;; this takes minutes and runs the JVM out of memory
;; without printing anything
(println (first semantically-also-combos))

根据type,它们都是clojure.lang.LazySeq,但一个按预期工作,另一个使进程崩溃。为什么要实现整个 seq 只是通过身份函数运行它并检查它是否为 nil?

完整的代码重现

(ns my-project.core
  (:gen-class))

;;; Copied from rosetta code
(defn combinations
  "If m=1, generate a nested list of numbers [0,n)
   If m>1, for each x in [0,n), and for each list in the recursion on [x+1,n), cons the two"
  [m n]
  (letfn [(comb-aux
            [m start]
            (if (= 1 m)
              (for [x (range start n)]
                (list x))
              (for [x (range start n)
                    xs (comb-aux (dec m) (inc x))]
                (cons x xs))))]
    (comb-aux m 0)))

(println "Generating combinations...")
(def combos (combinations 4 143))

(def should-also-be-combos
  (filter nil? (map identity combos)))

(defn -main
  "Calculates combos"
  [& _args]
  (println (type combos))
  (println (type should-also-be-combos)))

【问题讨论】:

    标签: clojure jvm out-of-memory


    【解决方案1】:

    您为什么不检查(filter nil? (map identity combos)) 是否确实返回了您对传递给combinations 的小得多的参数所期望的结果?我这样做了。这就是 combinations 使用参数 2 和 5 返回的内容:

    (combinations 2 5)
    ;; => ((0 1) (0 2) (0 3) (0 4) (1 2) (1 3) (1 4) (2 3) (2 4) (3 4))
    

    这就是我们通过示例的额外惰性序列操作得到的结果:

    (filter nil? (map identity (combinations 2 5)))
    ;; => ()
    

    (filter nil? ...) 所做的是保留所有满足谓词的元素。并且输入序列中没有一个元素是nil?,所以它会扫描整个输入序列并且找不到。也许您想使用(remove nil? ...),它将删除满足谓词的元素?

    (remove nil? (map identity (combinations 2 5)))
    ;; => ((0 1) (0 2) (0 3) (0 4) (1 2) (1 3) (1 4) (2 3) (2 4) (3 4))
    

    回到原来的例子,这是我使用remove得到的:

    (first (combinations 4 143))
    ;; => (0 1 2 3)
    
    (first (remove nil? (map identity (combinations 4 143))))
    ;; => (0 1 2 3)
    

    我的一般建议是先使用“较小”的数据(例如 2 和 5)测试您的函数,然后再使用“较大”的数据(例如 4 和 143)。

    【讨论】:

    • 你是对的,想要(not (nil? x)),愚蠢的错误。
    • 尝试 REPL 中的内容有助于发现这些错误。
    【解决方案2】:

    我将示例修改如下:

    (ns tst.demo.core
      (:use tupelo.core tupelo.test))
    
    (def cnt (atom 0))
    
    ; Copied from rosetta code
    (defn combinations
      "If m=1, generate a nested list of numbers [0,n)
       If m>1, for each x in [0,n), and for each list in the recursion on [x+1,n), cons the two"
      [m n]
      (let [comb-aux (fn comb-aux
                       [m start]
                       (swap! cnt inc)
                       (when (zero? (mod @cnt 100))
                         (print \.)
                         (flush))
                       (when (zero? (mod @cnt 10000))
                         (newline)
                         (print @cnt "  "))
                       (if (= 1 m)
                         (for [x (range start n)]
                           (list x))
                         (for [x  (range start n)
                               xs (comb-aux (dec m) (inc x))]
                           (cons x xs))))]
        (comb-aux m 0)))
    
    (println "Generating combinations...")
    (def combos (combinations 4 143))
    
    (def should-also-be-combos
      (filter nil? (map identity combos)))
    

    和调用:

    (dotest
      (newline)
      (spyx (type combos))
      (spyx (type should-also-be-combos))
      (newline)
      (println :1)
      (println (first combos))
      (newline)
      (println :2)
      (println (first should-also-be-combos))
      (newline))
    

    结果:

    -------------------------------
       Clojure 1.10.2    Java 15
    -------------------------------
    
    Testing tst.demo.core
    
    (type combos)                  => clojure.lang.LazySeq
    (type should-also-be-combos)   => clojure.lang.LazySeq
    
    :1
    (0 1 2 3)
    
    :2
    ....................................................................................................
    10000   ....................................................................................................
    20000   ....................................................................................................
    30000   ....................................................................................................
    40000   ....................................................................................................
    50000   ....................................................................................................
    60000   ....................................................................................................
    70000   ....................................................................................................
    80000   ....................................................................................................
    90000   ....................................................................................................
    100000   ....................................................................................................
    110000   ....................................................................................................
    120000   ....................................................................................................
    130000   ....................................................................................................
    140000   ....................................................................................................
    150000   ....................................................................................................
    160000   ....................................................................................................
    170000   ....................................................................................................
    180000   ....................................................................................................
    190000   ....................................................................................................
    200000   ....................................................................................................
    210000   ....................................................................................................
    220000   ....................................................................................................
    230000   ....................................................................................................
    240000   ....................................................................................................
    250000   ....................................................................................................
    260000   ....................................................................................................
    270000   ....................................................................................................
    280000   ....................................................................................................
    290000   ....................................................................................................
    300000   ....................................................................................................
    310000   ....................................................................................................
    320000   ....................................................................................................
    330000   ....................................................................................................
    340000   ....................................................................................................
    350000   ....................................................................................................
    360000   ....................................................................................................
    370000   ....................................................................................................
    380000   ....................................................................................................
    390000   ....................................................................................................
    400000   ....................................................................................................
    410000   ....................................................................................................
    420000   ....................................................................................................
    430000   ....................................................................................................
    440000   ....................................................................................................
    450000   ....................................................................................................
    460000   ....................................................................................................
    470000   ....................................................................................................
    480000   ..........................................................................nil
    

    在我的台式电脑上计时(5 岁,8 核,32Gb RAM):

    Passed all tests
    Finished at 16:37:28.484 (run time: 5.354s)
    

    所以你可以看到该函数被调用了大约 1/2 百万次,这在我的计算机上需要 5.3 秒。

    我相信您看到的内容与Clojure Don'ts: Concat 中描述的内容相似。它的递归形式也让我想起了著名的Ackerman Function,这是一个看似无害的函数即使对于小的输入值也可能“爆炸”的例子。

    【讨论】:

    • 是的,它在大约 60 秒内耗尽了我微不足道的 macbook air 的内存。那篇文章似乎是一个不错的起点,谢谢!
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2015-01-13
    • 2011-03-15
    • 1970-01-01
    相关资源
    最近更新 更多