考虑到线程数，如何正确安排 boost::wait_for_any 中的任务？

Question

我有以下一段代码，它将异步触发推送到任务向量上的每个已完成任务的回调

#define BOOST_THREAD_PROVIDES_FUTURE
#define BOOST_THREAD_PROVIDES_FUTURE_CONTINUATION

#include <fstream>
#include <iostream>
#include <random>
#include <thread>
#include <vector>

#include <boost/thread/future.hpp>

using vbf = std::vector<boost::future<std::pair<std::string, int>>>;

void processFile(vbf& v, const std::string& f, int offset)
{
  v.emplace_back(boost::async([offset, &f]() {
    std::cout << "Starting " << f << '\n';
    std::ofstream ofs(f);
    const int max = offset * 1000000;
    for (int i = 0; i < max; ++i) ofs << i;
    // line below wouldn't present the real problem
    //std::this_thread::sleep_for(std::chrono::milliseconds(max));
    return std::make_pair(f, max);
  }));
}

void printProgress(vbf& tasks)
{
  auto it = boost::wait_for_any(tasks.begin(), tasks.end());
  while (it != tasks.end())
  {
    const auto res = it->get();
    std::cout << "processed file \"" << res.first << "\" with " << res.second << " records\n";
    tasks.erase(it);
    it = boost::wait_for_any(tasks.begin(), tasks.end());
  }
}

int main()
{
  std::vector<std::string> vs{
      "file1",  "file2",  "file3",  "file4",  "file5",  "file6",  "file7",  "file8",  "file9",
      "file10", "file11", "file12", "file13", "file14", "file15", "file16", "file17", "file18",
      "file19", "file20", "file21", "file22", "file23", "file24", "file25", "file26", "file27",
      "file28", "file29", "file30", "file31", "file32", "file33", "file34", "file35", "file36",
      "file37", "file38", "file39", "file40", "file41", "file42", "file43", "file44", "file45",
      "file46", "file47", "file48", "file49", "file50"};

  vbf v;
  v.reserve(vs.size());

  std::random_device rd;
  std::mt19937 gen(rd());
  std::uniform_int_distribution<> dis(1, 10);

  for (const auto& f : vs) processFile(v, f, dis(gen));

  printProgress(v);
}

这将通过从std::vector<std::string> vs 写入文件来“模拟”负载（sleep() 有注释，但这不会造成真正的问题，因为那时 CPU 上没有实际负载）。

我的问题是如何/使用什么方法可以正确安排/排队在这里完成的工作，这样我就不会发生争用、线程“争夺”CPU 等问题？

我意识到这是一个不平凡的问题，它可能有很多解决方案/方法。

在线IDEhttp://melpon.org/wandbox/permlink/aME5cUecefZaBof4

Answer 1

为避免争用，您可以做的第一件事是不要启动/运行（同时）多于执行它们的内核（超线程）的线程。