Skip to content

Commit

Permalink
Enable configuration of Go integration (#3)
Browse files Browse the repository at this point in the history
Also improve docs and temporarily use private cache mounts
  • Loading branch information
EricHripko authored Feb 23, 2021
1 parent ec27d2b commit 6df4510
Show file tree
Hide file tree
Showing 63 changed files with 3,432 additions and 43 deletions.
2 changes: 2 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
run:
# Exclude tests
tests: false
skip-files:
- tests
67 changes: 61 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# pack.yaml

[![Go Report](https://goreportcard.com/badge/github.com/EricHripko/pack.yaml)](https://goreportcard.com/report/github.com/EricHripko/pack.yaml)
![CI](https://github.com/EricHripko/pack.yaml/actions/workflows/ci.yml/badge.svg)

Expand All @@ -14,10 +15,64 @@ footprint.

`pack.yaml` takes advantage of the plugin system to provide deep integrations
with various language and build ecosystems. As of today, the following
functionality is available:
functionality is available.

### Go

```text
=> [internal] load build definition from pack.yaml 0.0s
=> => transferring dockerfile: 98B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 50B 0.0s
=> resolve image config for docker.io/erichripko/pack.yaml:latest 0.0s
=> CACHED docker-image://docker.io/erichripko/pack.yaml:latest 0.0s
=> [internal] load build definition from pack.yaml 0.0s
=> => transferring dockerfile: 98B 0.0s
=> [internal] load context 0.0s
=> => transferring context: 109.58kB 0.0s
=> load metadata for docker.io/library/golang:1.14 0.5s
=> load metadata for gcr.io/distroless/base:debug 0.5s
=> Base build image is golang:1.14 7.9s
=> => resolve docker.io/library/golang:1.14 0.0s
=> => sha256:0ecb575e629cd60aa802266a3bc6847dcf4073a 50.40MB / 50.40MB 0.0s
=> => sha256:feab2c490a3cea21cc051ff29c33cc9857418ed 10.00MB / 10.00MB 0.0s
=> => sha256:1517911a35d7939f446084c1d4c31afc552678e 68.72MB / 68.72MB 0.0s
=> => sha256:48bbd1746d63c372e12f884178053851d87f3 124.14MB / 124.14MB 0.0s
=> => sha256:1a7173b5b9a3af3e29a5837e0b2027e1c438fd1b8 2.36kB / 2.36kB 0.0s
=> => sha256:6a39a02f74ffee82a169f2d836134236dc6f69e59 1.79kB / 1.79kB 0.0s
=> => sha256:21a5635903d69da3c3d928ed429e3610eecdf878c 7.03kB / 7.03kB 0.0s
=> => sha256:7467d1831b6947c294d92ee957902c3cd448b17c5 7.83MB / 7.83MB 0.0s
=> => sha256:f15a0f46f8c38f4ca7daecf160ba9cdb3ddeafd 51.83MB / 51.83MB 0.0s
=> => sha256:944903612fdd2364b4647cf3c231c41103d1fd378add4 126B / 126B 0.0s
=> => extracting sha256:48bbd1746d63c372e12f884178053851d87f3ea4b415f3 7.1s
=> => extracting sha256:944903612fdd2364b4647cf3c231c41103d1fd378add43 0.0s
=> Base runtime image is gcr.io/distroless/base:debug 0.1s
=> Create build output directory 0.6s
=> Build github.com/mpppk/cli-template 8.8s
=> CACHED Create output directory 0.0s
=> Install application(s) 0.1s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:86d32b9ed657253d9713fa1d6e0859219fc607fbbca 0.0s
```

[Go](https://golang.org/) - runs `go build` on your project and takes full
advantage of the build cache for best performance. Integration supports the
following dependency management methods:

- [go mod](https://golang.org/ref/mod) - automatically picks up the version
of Go from `go.mod` and relies on module cache for best performance.

The following additional configuration is supported by the integration:

- [Go](https://golang.org/) - runs `go build` on your project and takes full
advantage of the build cache for best performance. Following dependency
management methods are supported:
- [go mod](https://golang.org/ref/mod) - automatically picks up the version
of Go from `go.mod` and relies on module cache for best performance.
```yaml
# syntax = erichripko/pack.yaml
go:
# Version of Go to use for the project.
version: "1.14"
# List of Go build tags to set.
tags: ["wireinject"]
# How the dependencies are specified.
# Supported values are: modules
dependencyMode: modules
```
3 changes: 3 additions & 0 deletions codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Exclude tests
ignore:
- test
89 changes: 68 additions & 21 deletions pkg/plugins/golang/golang.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ import (
"context"
"fmt"
"regexp"
"strings"

"github.com/EricHripko/pack.yaml/pkg/cib"
"github.com/EricHripko/pack.yaml/pkg/packer2llb"
"github.com/mitchellh/mapstructure"

"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb"
Expand All @@ -15,6 +17,7 @@ import (
"github.com/pkg/errors"
fsutil "github.com/tonistiigi/fsutil/types"
"golang.org/x/mod/modfile"
"golang.org/x/sync/errgroup"
)

// Regular expression for detecting a Go project.
Expand All @@ -27,31 +30,55 @@ var (
)

// DependencyMode describes all the supported methods for dependency resolution.
type DependencyMode int
type DependencyMode string

const (
// DMUnknown represents an unrecognised dependency method.
DMUnknown = iota
DMUnknown = "unknown"
// DMGoMod represents a go.mod/go.sum project.
DMGoMod
DMGoMod = "modules"
)

// Config for the Go plugin.
type Config struct {
// Version of Go used.
Version string
// Method for declaring dependencies.
DependencyMode DependencyMode
// Build tags.
Tags []string
}

// Plugin for Go ecosystem.
type Plugin struct {
// General configuration supplied by the user.
config *cib.Config
// Mode for dependency resolution.
dependencyMode DependencyMode
// Version of Go used.
version string
// Configuration for the plugin.
pluginConfig *Config
// Name of the project.
name string
}

// NewPlugin creates a new Go plugin with correct defaults.
func NewPlugin() *Plugin {
return &Plugin{
config: cib.NewConfig(),
pluginConfig: &Config{
DependencyMode: DMUnknown,
},
}
}

// Detect if this is a Go project and identify the context.
func (p *Plugin) Detect(ctx context.Context, src client.Reference, config *cib.Config) error {
// Save config
p.config = config
if other, ok := p.config.Other["go"]; ok {
err := mapstructure.Decode(other, p.pluginConfig)
if err != nil {
return err
}
}

// Look for go files
err := cib.WalkRecursive(ctx, src, func(file *fsutil.Stat) error {
Expand All @@ -65,27 +92,40 @@ func (p *Plugin) Detect(ctx context.Context, src client.Reference, config *cib.C
}

// Identify dependency method
goMod, err := src.ReadFile(ctx, client.ReadRequest{Filename: "go.mod"})
if err == nil {
_, err = src.ReadFile(ctx, client.ReadRequest{Filename: "go.sum"})
if err == nil {
p.dependencyMode = DMGoMod
if p.pluginConfig.DependencyMode == DMUnknown {
goModGroup := new(errgroup.Group)
goModGroup.Go(func() error {
_, err := src.ReadFile(ctx, client.ReadRequest{Filename: "go.mod"})
return err
})
goModGroup.Go(func() error {
_, err := src.ReadFile(ctx, client.ReadRequest{Filename: "go.sum"})
return err
})
if err := goModGroup.Wait(); err == nil {
p.pluginConfig.DependencyMode = DMGoMod
}
}

// Pick up the project context from the dependency metadata
switch p.dependencyMode {
switch p.pluginConfig.DependencyMode {
case DMUnknown:
return ErrUnknownDep
case DMGoMod:
goMod, err := modfile.ParseLax("go.mod", goMod, nil)
data, err := src.ReadFile(ctx, client.ReadRequest{Filename: "go.mod"})
if err != nil {
return errors.Wrap(err, "fail to read go.mod")
}
goMod, err := modfile.ParseLax("go.mod", data, nil)
if err != nil {
return errors.Wrap(err, "fail to parse go.mod")
}
if goMod.Go == nil || goMod.Module == nil {
return ErrModIncomplete
}
p.version = goMod.Go.Version
if p.pluginConfig.Version == "" {
p.pluginConfig.Version = goMod.Go.Version
}
p.name = goMod.Module.Mod.Path
}

Expand All @@ -106,7 +146,7 @@ const (
// Build the image for this Go project.
func (p *Plugin) Build(ctx context.Context, platform *specs.Platform, build cib.Service) (*llb.State, *dockerfile2llb.Image, error) {
// Choose base image
base := "golang:" + p.version
base := "golang:" + p.pluginConfig.Version
state, _, err := build.From(
base,
platform,
Expand All @@ -127,27 +167,34 @@ func (p *Plugin) Build(ctx context.Context, platform *specs.Platform, build cib.
llb.WithCustomName("Create build output directory"),
)
// Build
args := []string{"go", "install", "-v"}
if len(p.pluginConfig.Tags) > 0 {
args = append(args, "-tags")
args = append(args, strings.Join(p.pluginConfig.Tags, ","))
}
args = append(args, "./...")

run := []llb.RunOption{
// Mount source code
llb.AddMount(dirSrc, src, llb.Readonly),
// Install executables
llb.AddEnv("GOBIN", dirInstall),
llb.Args([]string{"go", "install", "-v", "./..."}),
llb.Args(args),
// Cache build outputs
llb.AddMount(
dirGoBuildCache,
llb.Scratch(),
llb.AsPersistentCacheDir("go-build", llb.CacheMountShared),
llb.AsPersistentCacheDir("go-build", llb.CacheMountPrivate),
),
llb.AddEnv("GOCACHE", dirGoBuildCache),
llb.WithCustomNamef("Build %s", p.name),
}
if p.dependencyMode == DMGoMod {
if p.pluginConfig.DependencyMode == DMGoMod {
// Cache modules
run = append(run, llb.AddMount(
dirGoModCache,
llb.Scratch(),
llb.AsPersistentCacheDir("go-mod", llb.CacheMountShared),
llb.AsPersistentCacheDir("go-mod", llb.CacheMountPrivate),
))
}
buildState := state.Dir(dirSrc).Run(run...).Root()
Expand Down Expand Up @@ -185,5 +232,5 @@ func (p *Plugin) Build(ctx context.Context, platform *specs.Platform, build cib.

func init() {
// Register the plugin with the frontend.
packer2llb.Register(&Plugin{})
packer2llb.Register(NewPlugin())
}
Loading

0 comments on commit 6df4510

Please sign in to comment.