多线程计算均值和标准不提高效率

Question

我是 C++ 多线程编程领域的新手，我尝试使用多线程并行计算数据的均值和标准差，以减少时间成本。我计算平均值和标准差的函数如下。

void cal_mean_std(float* data, float* mean, float* sd, int N, int start_index, int span_cols)
{
    int value;
    for(int j = start_index; j < start_index + span_cols; j++){
        mean[j] = 0;
        sd[j] = 0;
        for (int i = 0; i < N; i++) {
            value = data[j * N + i];
            mean[j] += value;
            sd[j] += value * value;
        }
        mean[j] = mean[j] / N;
        sd[j] = sqrt(sd[j] / N - mean[j] * mean[j]);
    }
}

我指定每个线程的起始索引和计算范围，并按以下方式激活我的线程池。

    x.mean = new float[x.M];
    x.sd = new float[x.M];
    std::vector<std::thread> thread_pool;

    int h = 4;
    thread_pool.reserve(h);
    int SNIPs = static_cast<int>(x.M / h + 1);
    int SNIPs_final = x.M - (h - 1) * SNIPs;
     for (int i = 0; i < h - 1; i++)
     {
         thread_pool.push_back(std::thread(std::bind(cal_mean_std, x.data, x.mean, x.sd,
                                                 x.N, i*SNIPs, SNIPs)));
     }
    thread_pool.push_back(std::thread(std::bind(cal_mean_std, x.data, x.mean, x.sd,
                                                 x.N, (h-1)*SNIPs, SNIPs_final)));
    for (int i = 0; i < h; i++)
        thread_pool.at(i).join();

x.M 是我的数据的总列数。但是，我发现以这种方式实现并没有提高程序效率。我不确定问题是什么。

实际上，我们可以模拟数据来进行计算。我的数据大小是 5k x 300k。一个线程对所有数据使用for循环的顺序计算需要15秒。我的多线程版本有时需要 16 秒。

仿真代码如下，我发现当我使用h = 1时，程序需要6s才能完成。但是，当我使用 h = 4 时，程序需要 14 秒才能完成。

#include <thread>
#include <vector>
#include <stdlib.h>
#include <vector>
#include <stdio.h>
#include <iostream>
#include <math.h>

void gen_matrix(int N, int P, float* data){
    for (int i = 0; i < N * P; i++)
    {
        data[i] = rand() % 10;
    }
}

void cal_mean_std(float* data, float* mean, float* sd, int N, int start_index, int span_cols)
{
    int value;
    for(int j = start_index; j < start_index + span_cols; j++){
        mean[j] = 0;
        sd[j] = 0;
        for (int i = 0; i < N; i++) {
            value = data[j * N + i];
            mean[j] += value;
            sd[j] += value * value;
        }
        mean[j] = mean[j] / N;
        sd[j] = sqrt(sd[j] / N - mean[j] * mean[j]);
    }
}

int main()
{
    int N = 5000;
    int P = 300000;
    float* data = new float[N*P];
    gen_matrix(N, P, data);
    float* mean = new float[P];
    float* std = new float[P];
    std::vector<std::thread> thread_pool;
    clock_t t1;
    t1 = clock();
    int h = 1;
    thread_pool.reserve(h);
    int SNIPs = static_cast<int>(P / h + 1);
    int SNIPs_final = P - (h - 1) * SNIPs;
    for (int i = 0; i < h - 1; i++)
    {
        thread_pool.push_back(std::thread(std::bind(cal_mean_std, data, mean, std,
                                                    N, i*SNIPs, SNIPs)));
    }
    thread_pool.push_back(std::thread(std::bind(cal_mean_std, data, mean, std,
                                                N, (h-1)*SNIPs, SNIPs_final)));
    for (int i = 0; i < h; i++)
        thread_pool.at(i).join();
    std::cout <<"Time for the cal mean and std is " << (clock() - t1) * 1.0/CLOCKS_PER_SEC << std::endl;
    return 0;
}

Answer 1

谢谢大家。最后，我发现我的代码有什么问题。计时器clock_t 计算 CPU 消耗时间而不是挂起时间。