使用 docker-compose 安装包

Question

在 docker-compose 的构建过程中寻找安装 Go 包的帮助。特别是去科利。

设置：

docker-compose.yml

services:
  crawler:
    container_name: crawler
    build: ./crawler/
    working_dir: /go/src/crawler
    volumes:
      - ./data:/go/src/crawler/data
      - ./crawler:/go/src/crawler

Dockerfile

FROM golang:1.7-alpine

ADD . /go/src/crawler
WORKDIR /go/src/crawler

RUN \
       apk add --no-cache bash git openssh && \
       go get github.com/golang/example/stringutil/... && \
       go get github.com/gocolly/colly/... && \
       go install

CMD ["go","run","collect.go"]

package main

import (
    "fmt"
    "github.com/golang/example/stringutil"
        "github.com/gocolly/colly/v2"
)

func main() {
    fmt.Println(stringutil.Reverse("!selpmaxe oG ,olleH"))
    c := colly.NewCollector()

    // Find and visit all links
    c.OnHTML("a[href]", func(e *colly.HTMLElement) {
        e.Request.Visit(e.Attr("href"))
    })

    c.OnRequest(func(r *colly.Request) {
        fmt.Println("Visiting", r.URL)
    })

    c.Visit("http://go-colly.org/")

}

如果我在没有 Colly 的情况下运行项目，即。只需"github.com/golang/example/stringutil" 然后项目安装包并正确运行，但是使用 Colly 我得到：

OK: 34 MiB in 25 packages
package github.com/gocolly/colly
    imports github.com/gocolly/colly/v2/debug: cannot find package "github.com/gocolly/colly/v2/debug" in any of:
    /usr/local/go/src/github.com/gocolly/colly/v2/debug (from $GOROOT)
    /go/src/github.com/gocolly/colly/v2/debug (from $GOPATH)
package github.com/gocolly/colly
    imports github.com/gocolly/colly/v2/storage: cannot find package "github.com/gocolly/colly/v2/storage" in any of:
    /usr/local/go/src/github.com/gocolly/colly/v2/storage (from $GOROOT)
    /go/src/github.com/gocolly/colly/v2/storage (from $GOPATH)
package github.com/gocolly/colly/extensions
    imports github.com/gocolly/colly/v2: cannot find package "github.com/gocolly/colly/v2" in any of:
    /usr/local/go/src/github.com/gocolly/colly/v2 (from $GOROOT)
    /go/src/github.com/gocolly/colly/v2 (from $GOPATH)
ERROR: Service 'crawler' failed to build: The command '/bin/sh -c apk add --no-cache bash git openssh &&        go get github.com/golang/example/stringutil/... &&        go get github.com/gocolly/colly/... &&        go install' returned a non-zero code: 1

我最初的想法是安装- ./crawler:/go/src/crawler 会覆盖包，但后来我对为什么github.com/golang/example/stringutil 工作正常感到困惑。

Answer 1

基本上你的问题是你引用了一个模块路径（v2 泄露了它），在撰写本文时是seems to be broken。

但是，通过一些调整和更正，您的项目可以运行：

main.go

package main

import (
    "fmt"
    // Note the version suffix "/v2" is gone
    // This installs the v1.2.0 version, but apparently, this works, too
    "github.com/gocolly/colly"
    "github.com/golang/example/stringutil"
)

func main() {
    fmt.Println(stringutil.Reverse("!selpmaxe oG ,olleH"))

    // Limit the domains for the sake of the example
    c := colly.NewCollector(colly.AllowedDomains("go-colly.org"))

    // Find and visit all links
    c.OnHTML("a[href]", func(e *colly.HTMLElement) {
        e.Request.Visit(e.Attr("href"))
    })

    c.OnRequest(func(r *colly.Request) {
        fmt.Println("Visiting", r.URL)
    })

    c.Visit("http://go-colly.org/")

}

现在只需调用

go mod init stackoverflow.com/user3939059/crawler

（当然要调整域名和用户名）

Dockerfile

虽然您可以在基于“golang”映像的 Docker 映像中执行 go run main.go，但这简直是低效的。最好静态编译您的应用程序，您甚至可以使用空的临时映像作为基础来运行您的应用程序：

FROM golang:alpine as builder

ADD . /go/src/crawler
WORKDIR /go/src/crawler
RUN \
       apk add --no-cache tzdata && \
       adduser -D -g '' appuser

# Statically compile our application
RUN GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -ldflags="-w -s" -a -installsuffix cgo -o /crawler


FROM scratch
# Import the files required by the standard library from builder.
COPY --from=builder /usr/share/zoneinfo /usr/share/zoneinfo
COPY --from=builder /etc/ssl/certs/ca-certificates.crt  /etc/ssl/certs/

# Make sure the user can be identified by the docker daemon
COPY --from=builder /etc/passwd /etc/passwd
# Copy our static executable
COPY --from=builder /crawler /bin/crawler
# Use an unprivileged user.
USER appuser
# Run the binary.
CMD ["/bin/crawler"]

通过上述更改并在docker build -t user3939059/crawler:latest . 之后，您可以运行映像：

$ docker run --rm user3939059/crawler
Hello, Go examples!
Visiting http://go-colly.org/
Visiting http://go-colly.org/docs/
Visiting http://go-colly.org/articles/
Visiting http://go-colly.org/services/
Visiting http://go-colly.org/datasets/
Visiting http://go-colly.org/data/factbase_transcripts.tar.bz2
Visiting http://go-colly.org/docs/examples/factbase/
Visiting http://go-colly.org/docs/introduction/install/
Visiting http://go-colly.org/docs/introduction/start/
Visiting http://go-colly.org/docs/introduction/configuration/
Visiting http://go-colly.org/docs/best_practices/debugging/
Visiting http://go-colly.org/docs/best_practices/distributed/
Visiting http://go-colly.org/docs/best_practices/storage/
Visiting http://go-colly.org/docs/best_practices/multi_collector/
Visiting http://go-colly.org/docs/best_practices/crawling/
Visiting http://go-colly.org/docs/best_practices/extensions/
Visiting http://go-colly.org/docs/examples/basic/
Visiting http://go-colly.org/docs/examples/error_handling/
Visiting http://go-colly.org/docs/examples/login/
Visiting http://go-colly.org/docs/examples/max_depth/
Visiting http://go-colly.org/docs/examples/multipart/
Visiting http://go-colly.org/docs/examples/parallel/
Visiting http://go-colly.org/docs/examples/proxy_switcher/
Visiting http://go-colly.org/docs/examples/queue/
Visiting http://go-colly.org/docs/examples/random_delay/
Visiting http://go-colly.org/docs/examples/rate_limit/
Visiting http://go-colly.org/docs/examples/redis_backend/
Visiting http://go-colly.org/docs/examples/request_context/
Visiting http://go-colly.org/docs/examples/scraper_server/
Visiting http://go-colly.org/docs/examples/url_filter/
Visiting http://go-colly.org/docs/examples/cryptocoinmarketcap/
Visiting http://go-colly.org/docs/examples/coursera_courses/
Visiting http://go-colly.org/docs/examples/google_groups/
Visiting http://go-colly.org/docs/examples/hackernews_comments/
Visiting http://go-colly.org/docs/examples/instagram/
Visiting http://go-colly.org/docs/examples/openedx_courses/
Visiting http://go-colly.org/docs/examples/reddit/
Visiting http://go-colly.org/docs/examples/shopify_sitemap/
Visiting http://go-colly.org/docs/examples/xkcd_store/
Visiting http://go-colly.org/contact/
Visiting http://go-colly.org/sitemap.xml
Visiting http://go-colly.org/docs/examples/coursera_courses
Visiting http://go-colly.org/docs/examples/redis_backend
Visiting http://go-colly.org/docs/best_practices/storage
Visiting http://go-colly.org/docs/examples/scraper_server
Visiting http://go-colly.org/data/cryptocoin_market_cap.csv
Visiting http://go-colly.org/data/coursera_courses.json
Visiting http://go-colly.org/data/xkcd_store_items.csv
Visiting http://go-colly.org/contact
Visiting http://go-colly.org/articles/first_release/
Visiting http://go-colly.org/articles/scraping_related_http_headers/
Visiting http://go-colly.org/articles/nonprofit/
Visiting http://go-colly.org/articles/how_to_scrape_instagram/
Visiting http://go-colly.org/articles/scraping_tips/