使用 Pytorch 加速 PSO 算法？

Question

我已经使用 PyTorch 实现了一个 PSO 算法，torch.cuda.FloatTensor。 我设法将它加快了 2 倍，但我期待的不止于此。 它认为检查限制 (line 41-55) 和参数更新 (line 58-72) 的步骤是原因。有一个“GPU”这样做吗？

这是我的代码：

import torch
import numpy as np
import matplotlib.pyplot as plt
dtype = torch.cuda.FloatTensor

def fitness(x,y):
    return -torch.abs(torch.sin(x)*torch.cos(y)*torch.exp(torch.abs(1-(torch.sqrt(x**2+y**2))/(3.1415))))

def velocity(v, gxbest, pxbest, pybest, x, pop, w, c1, c2):
    return w*torch.rand(pop).type(dtype)*v + \
       c1*torch.rand(pop).type(dtype)*(pxbest - x) + \
       c2*torch.rand(pop).type(dtype)*(gxbest.expand(x.size(0)) - x)

def PSO(pop, xmax, xmin, niter, wmax, wmin, c1, c2):

    vx                 = torch.rand(pop).type(dtype)
    vy                 = torch.rand(pop).type(dtype)
    best               = np.zeros(niter)
    x                 = (xmax -  xmin)*torch.rand(pop).type(dtype) + xmin
    y                 = (xmax -  xmin)*torch.rand(pop).type(dtype) + xmin
    z                 = fitness(x,y)
    [minz, indexminz] = z.min(0)
    gxbest            = x[indexminz] 
    gybest            = y[indexminz] 
    pxbest            = x
    pybest            = y
    pzbest            = z

    for K in range(niter):
        w      = wmax - ((wmax - wmin) / niter) * (K)
        vnextx = velocity(vx, gxbest, pxbest, pybest, x, pop, w, c1, c2)
        xnext  = x + vnextx
        vnexty = velocity(vy, gxbest, pxbest, pybest, y, pop, w, c1, c2)
        ynext  = y + vnexty

        **(41)** xnext = xnext.cpu()
        ynext = ynext.cpu()
        idxmax        = (xnext > xmax) # elements that are bigger that upper limit
        idxmim        = (xnext < xmin)  # elements that are smaller that upper limit
        xnext[idxmax] = xmax
        xnext[idxmim] = xmin
        idymax        = (ynext > xmax) # elements that are bigger that upper limit
        idymim        = (ynext < xmin)  # elements that are smaller that upper limit
        ynext[idymax] = xmax
        ynext[idymim] = xmin

        xnext = xnext.cuda() 
        **(55)** ynext = ynext.cuda() 

        znext = fitness(xnext,ynext)

        **(58)**[minznext, indexminznext]  = znext.min(0)

        if (minznext[0] < minz[0]):
            minz   = minznext
            gxbest = xnext[indexminznext]
            gybest = ynext[indexminznext]

        indexpbest         = (znext < pzbest)
        pxbest[indexpbest] = xnext[indexpbest]
        pybest[indexpbest] = ynext[indexpbest]
        pzbest[indexpbest] = znext[indexpbest]
        x                  = xnext
        y                  = ynext
        vx                 = vnextx
        **(72)** vy                 = vnexty

        best[K] = minz.cpu().numpy()
    return gxbest, gybest , minz, best

def main():

    pop, xmax, xmin, niter = 10000, 10, -10, 10
    wmax = 0.9
    wmin = 0.4
    c1   = 2.05
    c2   = 2.05
    xbest, ybest, fitbest, best = PSO(pop, xmax, xmin, niter, wmax, wmin, c1, c2)
    print(xbest)
    print(ybest)
    print(fitbest)
    t = np.linspace(0,niter,niter)
    plt.plot(t, best, 'k.-')
    plt.show()

main()

非常感谢您的帮助。

Answer 1

好吧好吧；使用 GPU 对我有用！

8.0557 [torch.cuda.FloatTensor 大小为 1 (GPU 0)]

9.6666 [torch.cuda.FloatTensor 大小为 1 (GPU 0)]

-19.2107 [torch.cuda.FloatTensor 大小为 1 (GPU 0)]

Answer 2

除了加速 PSO 之外，性能很大程度上取决于初始化以及您使用的 PSO 变体。

对于初始化，我只在值范围内随机设置粒子位置，并将初始速度设置为零，因为它们将在第一次和后续迭代期间更新。

大多数人现在至少使用 Clerc 的 constricted PSO 版本。看来，既然您有 wmin 和 wmax 值，那就是原始 PSO。

原始 PSO 使用：

w = wmax - (wmax - wmin) * (iter / numIterations)
v(j, part) = w * v(j, part) + c1 * random1 * (b(j, part) - p(j, part)) + c2 * random2 * (bg(j) - p(j, part))
If v(j, part) < vmin Then v(j, part) = vmin
If v(j, part) > vmax Then v(j, part) = vmax

其中j是维度之一，part是当前粒子，v(j, part)是速度，p(j, part)是粒子位置，c1和c2是社会参数，random1和random2 是随机统一数，bg(j) 是维度j 的最佳全局位置，vmin 和vmax 分别是0.1 和-0.1。但是，您可以根据每个特征值的范围更改vmin 和vmax。

受限 PSO 用途：

c1 = 2.05
c2 = 2.05
varphi = c1 + c2
kappa = 2 / Math.Abs(2 - varphi - Math.Sqrt(varphi * varphi - 4 * varphi))

v(j, part) = kappa * (p(j, part) + c1 * random1 * (p(j, part) - p(j, part)) + c2 * random2 * (bg(j) - p(j, part)))
If v(j, part) < vmin Then v(j, part) = vmin
If v(j, part) > vmax Then v(j, part) = vmax