Perl 中并行处理的最佳模块是什么？答案

【问题标题】：What is best module for parallel processing in Perl?Perl 中并行处理的最佳模块是什么？
【发布时间】：2013-10-30 19:08:52
【问题描述】：

Perl 中并行处理的最佳模块是什么？我从来没有在 Perl 中做过并行处理。

什么是好的 Perl 模块用于并行进程，将用于 DB 访问和邮件？

我查看了模块Parallel::ForkManager。任何想法表示赞赏。

【问题讨论】：

您想并行化 CPU 密集型或 IO 密集型代码吗？

标签： perl parallel-processing

【解决方案1】：

嗯，这一切都取决于具体情况。根据您确切需要实现的目标，您可能会更好地选择以下之一：

对于某些任务，还有专门的模块 - 例如LWP::Parallel::UserAgent，这基本上意味着 - 您必须向我们提供更多关于您想要实现的目标的详细信息以获得最佳答案。

【讨论】：

【解决方案2】：

Parallel::ForkManager，正如 POD 所说，可以限制分叉的进程数量。然后，您可以使用孩子们做任何工作。我记得使用该模块作为下载器。

【讨论】：

【解决方案3】：

Many-core Engine for Perl (MCE) 已发布在 CPAN 上。

http://code.google.com/p/many-core-engine-perl/

https://metacpan.org/module/MCE

MCE 附带了各种示例，展示了将诸如 cat（尝试使用 -n）之类的东西并行化到 greping 模式和字数聚合的实际用例场景。

barrier_sync.pl
          A barrier sync demonstration.

cat.pl    Concatenation script, similar to the cat binary.
egrep.pl  Egrep script, similar to the egrep binary.
wc.pl     Word count script, similar to the wc binary.

findnull.pl
          A parallel driven script to report lines containing
          null fields. It is many times faster than the binary
          egrep command. Try against a large file containing
          very long lines.

flow_model.pl
          Demonstrates MCE::Flow, MCE::Queue, and MCE->gather.

foreach.pl, forseq.pl, forchunk.pl
          These take the same sqrt example from Parallel::Loops
          and measures the overhead of the engine. The number
          indicates the size of @input which can be submitted
          and results displayed in 1 second.

          Parallel::Loops:     600  Forking each @input is expensive
          MCE foreach....:  34,000  Sends result after each @input
          MCE forseq.....:  70,000  Loops through sequence of numbers
          MCE forchunk...: 480,000  Chunking reduces overhead

interval.pl
          Demonstration of the interval option appearing in MCE 1.5.

matmult/matmult_base.pl, matmult_mce.pl, strassen_mce.pl
          Various matrix multiplication demonstrations benchmarking
          PDL, PDL + MCE, as well as parallelizing Strassen
          divide-and-conquer algorithm. Also included are 2 plain
          Perl examples.

scaling_pings.pl
          Perform ping test and report back failing IPs to
          standard output.

seq_demo.pl
          A demonstration of the new sequence option appearing
          in MCE 1.3. Run with seq_demo.pl | sort

tbray/wf_mce1.pl, wf_mce2.pl, wf_mce3.pl
          An implementation of wide finder utilizing MCE.
          As fast as MMAP IO when file resides in OS FS cache.
          2x ~ 3x faster when reading directly from disk.

【讨论】：

【解决方案4】：

您还可以查看线程，Coro、Parallel::Iterator。

【讨论】：