【问题标题】:In golang, why does my program run slower when I use a buffered (asynchronous) channel?在 golang 中,当我使用缓冲(异步)通道时,为什么我的程序运行速度变慢?
【发布时间】:2013-05-18 02:25:45
【问题描述】:

我对 golang 还很陌生,所以我确信这个问题很大程度上是由于我的一些概念上的缺陷。

在 golang 中,我们可以有两种类型的通道:无缓冲和有缓冲(分别为同步和异步)。

unbufferedChan := make(chan string)
bufferedChan := make(chan string, 100)

通过无缓冲通道进行通信的两个 goroutine 必须相互等待。即接收 goroutine 阻塞直到发送者发送,发送者阻塞直到接收者接收。

在缓冲的情况下,仅当通道为空时接收器才会阻塞。仅当通道已满时,发送方才会阻塞。

通过使用缓冲通道,我希望减少 goroutine 阻塞的时间,从而提高代码的速度。相反,一切都变得更慢了。与我的预期相反(即“缓冲更高效”),无缓冲通道最终变得更快。

这是所有代码。 您需要先go get code.google.com/p/go-html-transform/html/transform

package main

import (
    "fmt"
    "math/rand"
    "os"
    "sync"
    "time"
    sel "code.google.com/p/go-html-transform/css/selector"
    h5 "code.google.com/p/go-html-transform/h5"
    gnhtml "code.google.com/p/go.net/html"
)

// Find a specific HTML element and return its textual element children.
func main() {
    test := `
<html>
  <head>
    <title>This is the test document!</title>
    <style>
      header: color=blue;
    </style>
  </head>
  <body>
    <div id="h" class="header">This is some text</div>
  </body>
</html>`

    // Get a parse tree for this HTML
    h5tree, err := h5.NewFromString(test)
    if err != nil {
        die(err)
    }
    n := h5tree.Top()

    // Create a Chain object from a CSS selector statement
    chn, err := sel.Selector("#h")
    if err != nil {
        die(err)
    }

    // Find the item.  Should be a div node with the text "This is some text"
    h := chn.Find(n)[0]

    // run our little experiment this many times total
    var iter int = 100000

    // When buffering, how large shall the buffer be?
    var bufSize uint = 100

    // Keep a running total of the number of times we've tried buffered
    //   and unbuffered channels.
    var bufCount int = 0
    var unbufCount int =0

    // Keep a running total of the number of nanoseconds that have gone by.
    var bufSum int64 = 0
    var unbufSum int64 = 0

    // Call the function {iter} times, randomly choosing whether to use a
    //   buffered or unbuffered channel.
    for i := 0; i < iter; i++ {
        if rand.Float32() < 0.5 {
            // No buffering
            unbufCount += 1
            startTime := time.Now()
            getAllText(h, 0)
            unbufSum += time.Since(startTime).Nanoseconds()     
        } else {
            // Use buffering
            bufCount += 1
            startTime := time.Now()
            getAllText(h, bufSize)
            bufSum += time.Since(startTime).Nanoseconds()
        }
    }
    unbufAvg := unbufSum / int64(unbufCount)
    bufAvg := bufSum / int64(bufCount)

    fmt.Printf("Unbuffered average time (ns): %v\n", unbufAvg)
    fmt.Printf("Buffered average time (ns): %v\n", bufAvg)
}

// Kill the program and report the error
func die(err error) {
    fmt.Printf("Terminating: %v\n", err.Error())
    os.Exit(1)
}

// Walk through all of a nodes children and construct a string consisting
//   of c.Data where c.Type == TextNode
func getAllText(n *gnhtml.Node, bufSize uint) string {
    var texts chan string
    if bufSize == 0 {
        // unbuffered, synchronous
        texts = make(chan string)
    } else {
        // buffered, asynchronous
        texts = make(chan string, bufSize)
    }

    wg := sync.WaitGroup{}


    // Go walk through all n's child nodes, sending only textual data
    //   over the texts channel.
    wg.Add(1)
    nTree := h5.NewTree(n)
    go func () {
        nTree.Walk(func (c *gnhtml.Node) {
            if c.Type == gnhtml.TextNode {
                texts <- c.Data
            }
        })

        close(texts)
        wg.Done()
    }()

    // As text data comes in over the texts channel, build up finalString
    wg.Add(1)
    finalString := ""
    go func () {
        for t := range texts {
            finalString += t
        }

        wg.Done()
    }()

    // Return finalString once both of the goroutines have finished.
    wg.Wait()
    return finalString
}

程序运行10次,结果如下:

Buffered average time (ns): 32088
Buffered average time (ns): 32183
Buffered average time (ns): 35091
Buffered average time (ns): 35798
Buffered average time (ns): 36966
Buffered average time (ns): 38707
Buffered average time (ns): 39464
Buffered average time (ns): 40021
Buffered average time (ns): 41063
Buffered average time (ns): 46248
Unbuffered average time (ns): 18265
Unbuffered average time (ns): 18804
Unbuffered average time (ns): 20268
Unbuffered average time (ns): 20401
Unbuffered average time (ns): 21652
Unbuffered average time (ns): 22630
Unbuffered average time (ns): 22907
Unbuffered average time (ns): 23326
Unbuffered average time (ns): 24133
Unbuffered average time (ns): 27546

【问题讨论】:

    标签: performance go channel


    【解决方案1】:

    将测试中缓冲区的创建外部化:

    package main
    
    import (
        "fmt"
        "math/rand"
        "os"
        "sync"
        "time"
        sel "code.google.com/p/go-html-transform/css/selector"
        h5 "code.google.com/p/go-html-transform/h5"
        gnhtml "code.google.com/p/go.net/html"
    )
    
    // Find a specific HTML element and return its textual element children.
    func main() {
        test := `
    <html>
      <head>
        <title>This is the test document!</title>
        <style>
          header: color=blue;
        </style>
      </head>
      <body>
        <div id="h" class="header">This is some text</div>
      </body>
    </html>`
    
        // Get a parse tree for this HTML
        h5tree, err := h5.NewFromString(test)
        if err != nil {
            die(err)
        }
        n := h5tree.Top()
    
        // Create a Chain object from a CSS selector statement
        chn, err := sel.Selector("#h")
        if err != nil {
            die(err)
        }
    
        // Find the item.  Should be a div node with the text "This is some text"
        h := chn.Find(n)[0]
    
        // run our little experiment this many times total
        var iter int = 100000
    
        // When buffering, how large shall the buffer be?
        var bufSize uint = 100
    
        // Keep a running total of the number of times we've tried buffered
        //   and unbuffered channels.
        var bufCount int = 0
        var unbufCount int =0
    
        // Keep a running total of the number of nanoseconds that have gone by.
        var bufSum int64 = 0
        var unbufSum int64 = 0
    
        // Call the function {iter} times, randomly choosing whether to use a
        //   buffered or unbuffered channel.
        for i := 0; i < iter; i++ {
            if rand.Float32() < 0.5 {
                // No buffering
                unbufCount += 1
                texts := make(chan string)
                startTime := time.Now()
                getAllText(h, 0, texts)
                unbufSum += time.Since(startTime).Nanoseconds()     
            } else {
                // Use buffering
                bufCount += 1
                texts := make(chan string, bufSize)
                startTime := time.Now()
                getAllText(h, bufSize, texts)
                bufSum += time.Since(startTime).Nanoseconds()
            }
        }
        unbufAvg := unbufSum / int64(unbufCount)
        bufAvg := bufSum / int64(bufCount)
    
        fmt.Printf("Unbuffered average time (ns): %v\n", unbufAvg)
        fmt.Printf("Buffered average time (ns): %v\n", bufAvg)
    }
    
    // Kill the program and report the error
    func die(err error) {
        fmt.Printf("Terminating: %v\n", err.Error())
        os.Exit(1)
    }
    
    // Walk through all of a nodes children and construct a string consisting
    //   of c.Data where c.Type == TextNode
    func getAllText(n *gnhtml.Node, bufSize uint, texts chan string) string {
        wg := sync.WaitGroup{}
    
    
        // Go walk through all n's child nodes, sending only textual data
        //   over the texts channel.
        wg.Add(1)
        nTree := h5.NewTree(n)
        go func () {
            nTree.Walk(func (c *gnhtml.Node) {
                if c.Type == gnhtml.TextNode {
                    texts <- c.Data
                }
            })
    
            close(texts)
            wg.Done()
        }()
    
        // As text data comes in over the texts channel, build up finalString
        wg.Add(1)
        finalString := ""
        go func () {
            for t := range texts {
                finalString += t
            }
    
            wg.Done()
        }()
    
        // Return finalString once both of the goroutines have finished.
        wg.Wait()
        return finalString
    }
    

    有了这个(在启动计时器之前移动频道创建)我有这个结果:

    Buffered average time (ns): 2649
    Buffered average time (ns): 2655
    Buffered average time (ns): 2657
    Buffered average time (ns): 2695
    Buffered average time (ns): 2695
    Buffered average time (ns): 2699
    Buffered average time (ns): 2719
    Buffered average time (ns): 2724
    Buffered average time (ns): 2793
    Buffered average time (ns): 3101
    Unbuffered average time (ns): 2997
    Unbuffered average time (ns): 3005
    Unbuffered average time (ns): 3027
    Unbuffered average time (ns): 3031
    Unbuffered average time (ns): 3048
    Unbuffered average time (ns): 3062
    Unbuffered average time (ns): 3086
    Unbuffered average time (ns): 3092
    Unbuffered average time (ns): 3095
    Unbuffered average time (ns): 3191
    

    为了改进,解决方案可能是尽可能保持通道开放。

    【讨论】:

      【解决方案2】:

      实际上,我可能刚刚想通了。我认为额外的时间是由于创建了一个 100 元素的缓冲通道。

      我再次运行程序,这次使用的缓冲区大小为 10(仍然比我的小测试程序需要的多)而不是 100(原始缓冲区大小)。

      结果如下:

      Buffered average time (ns): 21930
      Buffered average time (ns): 22721
      Buffered average time (ns): 23011
      Buffered average time (ns): 23707
      Buffered average time (ns): 27701
      Buffered average time (ns): 28325
      Buffered average time (ns): 28851
      Buffered average time (ns): 29641
      Buffered average time (ns): 30417
      Buffered average time (ns): 32600
      Unbuffered average time (ns): 21077
      Unbuffered average time (ns): 21490
      Unbuffered average time (ns): 22332
      Unbuffered average time (ns): 22584
      Unbuffered average time (ns): 26438
      Unbuffered average time (ns): 26824
      Unbuffered average time (ns): 27322
      Unbuffered average time (ns): 27926
      Unbuffered average time (ns): 27985
      Unbuffered average time (ns): 30322
      

      这些数字更接近。

      【讨论】:

      • 请注意,当 GOMAXPROCS 设置为 1 时,您不会遇到巨大的性能提升,因为调度程序仅在锁定 IO 时调用。当没有任何阻塞时,不会调用另一侧的 go 例程,这意味着没有并行性。
      猜你喜欢
      • 2019-10-26
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2016-09-27
      • 1970-01-01
      • 2018-03-11
      • 2012-06-10
      • 1970-01-01
      相关资源
      最近更新 更多