Merge pull request #124 from tiborvass/update-docs

Update docs

.fossa.yml

@@ -0,0 +1,14 @@
+# Generated by FOSSA CLI (https://github.com/fossas/fossa-cli)
+# Visit https://fossa.com to learn more
+
+version: 2
+cli:
+  server: https://app.fossa.io
+  fetcher: custom
+  project: git@github.com:docker/buildx
+analyze:
+  modules:
+  - name: github.com/docker/buildx/cmd/buildx
+    type: go
+    target: github.com/docker/buildx/cmd/buildx
+    path: cmd/buildx

README.md

@@ -41,30 +41,21 @@ _buildx is Tech Preview_
 
 # Installing
 
-Using `buildx` as a docker CLI plugin requires using Docker 19.03.0 beta. A limited set of functionality works with older versions of Docker when invoking the binary directly.
+Using `buildx` as a docker CLI plugin requires using Docker 19.03. A limited set of functionality works with older versions of Docker when invoking the binary directly.
 
-### Docker Desktop (Edge)
+### Docker CE
 
-`buildx` is included with Docker Desktop Edge builds since 19.03.0-beta3.
+`buildx` comes bundled with Docker CE starting with 19.03, but requires experimental mode to be enabled on the Docker CLI.
-
+To enable it, `"experimental": "enabled"` can be added to the CLI configuration file `~/.docker/config.json`. An alternative is to set the `DOCKER_CLI_EXPERIMENTAL=enabled` environment variable.
-For more information see https://docs.docker.com/docker-for-mac/edge-release-notes/
-
-### Docker CE nightly builds
-
-`buildx` comes bundled with the Docker CE nightly builds. 
-- Mac: https://download.docker.com/mac/static/nightly/
-- Linux:
-```
-$ # uncomment next line to uninstall previous Docker CE installation if present
-$ # apt purge docker-ce docker-ce-cli
-$ curl -fsSL https://get.docker.com/ -o docker-install.sh
-$ CHANNEL=nightly sh docker-install.sh
-```
 
 ### Binary release
 
 Download the latest binary release from https://github.com/docker/buildx/releases/latest and copy it to `~/.docker/cli-plugins` folder with name `docker-buildx`.
 
+Change the permission to execute:
+```sh
+chmod a+x ~/.docker/cli-plugins/docker-buildx
+```
 
 After installing you can run `docker buildx` to see the new commands.
 
@@ -142,7 +133,7 @@ $ docker buildx build --platform linux/amd64,linux/arm64 .
 Finally, depending on your project, the language that you use may have good support for cross-compilation. In that case, multi-stage builds in Dockerfiles can be effectively used to build binaries for the platform specified with `--platform` using the native architecture of the build node. List of build arguments like `BUILDPLATFORM` and `TARGETPLATFORM` are available automatically inside your Dockerfile and can be leveraged by the processes running as part of your build.
 
 ```
-FROM --platform $BUILDPLATFORM golang:alpine AS build
+FROM --platform=$BUILDPLATFORM golang:alpine AS build
 ARG TARGETPLATFORM
 ARG BUILDPLATFORM
 RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM" > /log
@@ -340,21 +331,16 @@ Options:
 
 | Flag | Description |
 | --- | --- |
-| --append                | Append a node to builder instead of changing it
+| --append                 | Append a node to builder instead of changing it
-| --driver string         | Driver to use (eg. docker-container)
+| --buildkitd-flags string | Flags for buildkitd daemon
-| --leave                 | Remove a node from builder instead of changing it
+| --config string          | BuildKit config file
-| --name string           | Builder instance name
+| --driver string          | Driver to use (eg. docker-container)
-| --node string           | Create/modify node with given name
+| --driver-opt stringArray | Options for the driver
-| --platform stringArray  | Fixed platforms for current node
+| --leave                  | Remove a node from builder instead of changing it
-| --use                   | Set the current builder instance
+| --name string            | Builder instance name
-
+| --node string            | Create/modify node with given name
-#### `--driver DRIVER`
+| --platform stringArray   | Fixed platforms for current node
-
+| --use                    | Set the current builder instance
-Sets the builder driver to be used. There are two available drivers, each have their own specificities.
-
-- `docker` - Uses the builder that is built into the docker daemon. With this driver, the [`--load`](#--load) flag is implied by default on `buildx build`. However, building multi-platform images or exporting cache is not currently supported.
-
-- `docker-container` - Uses a buildkit container that will be spawned via docker. With this driver, both building multi-platform images and exporting cache are supported. However, images built will not automatically appear in `docker images` (see [`build --load`](#--load)).
 
 #### `--append`
 
@@ -368,6 +354,41 @@ $ docker buildx create --name eager_beaver --append mycontext2
 eager_beaver
 ```
 
+#### `--buildkitd-flags FLAGS`
+
+Adds flags when starting the buildkitd daemon. They take precedence over the configuration file specified by `--config`. See `buildkitd --help` for the available flags.
+
+Example:
+```
+--buildkitd-flags '--debug --debugaddr 0.0.0.0:6666'
+```
+
+#### `--config FILE`
+
+Specifies the configuration file for the buildkitd daemon to use. The configuration can be overridden by `--buildkitd-flags`. See an [example buildkitd configuration file](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md).
+
+#### `--driver DRIVER`
+
+Sets the builder driver to be used. There are two available drivers, each have their own specificities.
+
+- `docker` - Uses the builder that is built into the docker daemon. With this driver, the [`--load`](#--load) flag is implied by default on `buildx build`. However, building multi-platform images or exporting cache is not currently supported.
+
+- `docker-container` - Uses a buildkit container that will be spawned via docker. With this driver, both building multi-platform images and exporting cache are supported. However, images built will not automatically appear in `docker images` (see [`build --load`](#--load)).
+
+
+#### `--driver-opt OPTIONS`
+
+Passes additional driver-specific options. Details for each driver:
+
+- `docker` - No driver options
+- `docker-container`
+  - `image` - Sets the container image to be used for running buildkit.
+  - `network` - Sets the network mode for running the buildkit container.
+  - Example:
+    ```
+    --driver docker-container --driver-opt image=moby/buildkit:master,network=host
+    ```
+
 #### `--leave`
 
 Changes the action of the command to removes a node from a builder. The builder needs to be specified with `--name` and node that is removed is set with `--node`.

bake/bake.go

@@ -248,10 +248,10 @@ func (t *Target) normalize() {
 	t.Outputs = removeDupes(t.Outputs)
 }
 
-func TargetsToBuildOpt(m map[string]Target) (map[string]build.Options, error) {
+func TargetsToBuildOpt(m map[string]Target, noCache, pull bool) (map[string]build.Options, error) {
 	m2 := make(map[string]build.Options, len(m))
 	for k, v := range m {
-		bo, err := toBuildOpt(v)
+		bo, err := toBuildOpt(v, noCache, pull)
 		if err != nil {
 			return nil, err
 		}
@@ -260,7 +260,7 @@ func TargetsToBuildOpt(m map[string]Target) (map[string]build.Options, error) {
 	return m2, nil
 }
 
-func toBuildOpt(t Target) (*build.Options, error) {
+func toBuildOpt(t Target, noCache, pull bool) (*build.Options, error) {
 	if v := t.Context; v != nil && *v == "-" {
 		return nil, errors.Errorf("context from stdin not allowed in bake")
 	}
@@ -289,6 +289,8 @@ func toBuildOpt(t Target) (*build.Options, error) {
 		Tags:      t.Tags,
 		BuildArgs: t.Args,
 		Labels:    t.Labels,
+		NoCache:   noCache,
+		Pull:      pull,
 	}
 
 	platforms, err := platformutil.Parse(t.Platforms)

bake/compose.go

@@ -2,7 +2,9 @@ package bake
 
 import (
 	"fmt"
+	"os"
 	"reflect"
+	"strings"
 
 	"github.com/docker/cli/cli/compose/loader"
 	composetypes "github.com/docker/cli/cli/compose/types"
@@ -19,9 +21,22 @@ func parseCompose(dt []byte) (*composetypes.Config, error) {
 				Config: parsed,
 			},
 		},
+		Environment: envMap(os.Environ()),
 	})
 }
 
+func envMap(env []string) map[string]string {
+	result := make(map[string]string, len(env))
+	for _, s := range env {
+		kv := strings.SplitN(s, "=", 2)
+		if len(kv) != 2 {
+			continue
+		}
+		result[kv[0]] = kv[1]
+	}
+	return result
+}
+
 func ParseCompose(dt []byte) (*Config, error) {
 	cfg, err := parseCompose(dt)
 	if err != nil {
@@ -86,6 +101,8 @@ func toMap(in composetypes.MappingWithEquals) map[string]string {
 	for k, v := range in {
 		if v != nil {
 			m[k] = *v
+		} else {
+			m[k] = os.Getenv(k)
 		}
 	}
 	return m

build/build.go

@@ -24,6 +24,7 @@ import (
 	"github.com/moby/buildkit/client"
 	"github.com/moby/buildkit/session"
 	"github.com/moby/buildkit/session/upload/uploadprovider"
+	"github.com/moby/buildkit/util/entitlements"
 	"github.com/opencontainers/go-digest"
 	specs "github.com/opencontainers/image-spec/specs-go/v1"
 	"github.com/pkg/errors"
@@ -55,6 +56,7 @@ type Options struct {
 	CacheFrom []client.CacheOptionsEntry
 	CacheTo   []client.CacheOptionsEntry
 
+	Allow []entitlements.Entitlement
 	// DockerTarget
 }
 
@@ -324,11 +326,12 @@ func toSolveOpt(d driver.Driver, multiDriver bool, opt Options, dl dockerLoadCal
 	}
 
 	so := client.SolveOpt{
-		Frontend:      "dockerfile.v0",
+		Frontend:            "dockerfile.v0",
-		FrontendAttrs: map[string]string{},
+		FrontendAttrs:       map[string]string{},
-		LocalDirs:     map[string]string{},
+		LocalDirs:           map[string]string{},
-		CacheExports:  opt.CacheTo,
+		CacheExports:        opt.CacheTo,
-		CacheImports:  opt.CacheFrom,
+		CacheImports:        opt.CacheFrom,
+		AllowedEntitlements: opt.Allow,
 	}
 
 	if multiDriver {
@@ -454,6 +457,7 @@ func toSolveOpt(d driver.Driver, multiDriver bool, opt Options, dl dockerLoadCal
 	switch opt.NetworkMode {
 	case "host", "none":
 		so.FrontendAttrs["force-network-mode"] = opt.NetworkMode
+		so.AllowedEntitlements = append(so.AllowedEntitlements, entitlements.EntitlementNetworkHost)
 	case "", "default":
 	default:
 		return nil, nil, errors.Errorf("network mode %q not supported by buildkit", opt.NetworkMode)

build/entitlements.go

@@ -0,0 +1,21 @@
+package build
+
+import (
+	"github.com/moby/buildkit/util/entitlements"
+	"github.com/pkg/errors"
+)
+
+func ParseEntitlements(in []string) ([]entitlements.Entitlement, error) {
+	out := make([]entitlements.Entitlement, 0, len(in))
+	for _, v := range in {
+		switch v {
+		case "security.insecure":
+			out = append(out, entitlements.EntitlementSecurityInsecure)
+		case "network.host":
+			out = append(out, entitlements.EntitlementNetworkHost)
+		default:
+			return nil, errors.Errorf("invalid entitlement: %v", v)
+		}
+	}
+	return out, nil
+}

commands/bake.go

@@ -51,7 +51,7 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) error {
 		return nil
 	}
 
-	bo, err := bake.TargetsToBuildOpt(m)
+	bo, err := bake.TargetsToBuildOpt(m, in.noCache, in.pull)
 	if err != nil {
 		return err
 	}

commands/build.go

@@ -44,6 +44,8 @@ type buildOptions struct {
 	squash bool
 	quiet  bool
 
+	allow []string
+
 	// hidden
 	// untrusted   bool
 	// ulimits        *opts.UlimitOpt
@@ -84,8 +86,8 @@ func runBuild(dockerCli command.Cli, in buildOptions) error {
 			InStream:       os.Stdin,
 		},
 		Tags:        in.tags,
-		Labels:      listToMap(in.labels),
+		Labels:      listToMap(in.labels, false),
-		BuildArgs:   listToMap(in.buildArgs),
+		BuildArgs:   listToMap(in.buildArgs, true),
 		Pull:        in.pull,
 		NoCache:     in.noCache,
 		Target:      in.target,
@@ -167,6 +169,12 @@ func runBuild(dockerCli command.Cli, in buildOptions) error {
 	}
 	opts.CacheTo = cacheExports
 
+	allow, err := build.ParseEntitlements(in.allow)
+	if err != nil {
+		return err
+	}
+	opts.Allow = allow
+
 	return buildTargets(ctx, dockerCli, map[string]build.Options{"default": opts}, in.progress)
 }
 
@@ -214,6 +222,8 @@ func buildCmd(dockerCli command.Cli) *cobra.Command {
 
 	flags.StringVar(&options.target, "target", "", "Set the target build stage to build.")
 
+	flags.StringSliceVar(&options.allow, "allow", []string{}, "Allow extra privileged entitlement, e.g. network.host, security.insecure")
+
 	// not implemented
 	flags.BoolVarP(&options.quiet, "quiet", "q", false, "Suppress the build output and print image ID on success")
 	flags.StringVar(&options.networkMode, "network", "default", "Set the networking mode for the RUN instructions during build")
@@ -282,12 +292,16 @@ func commonFlags(options *commonOptions, flags *pflag.FlagSet) {
 	flags.BoolVar(&options.pull, "pull", false, "Always attempt to pull a newer version of the image")
 }
 
-func listToMap(values []string) map[string]string {
+func listToMap(values []string, defaultEnv bool) map[string]string {
 	result := make(map[string]string, len(values))
 	for _, value := range values {
 		kv := strings.SplitN(value, "=", 2)
 		if len(kv) == 1 {
-			result[kv[0]] = ""
+			if defaultEnv {
+				result[kv[0]] = os.Getenv(kv[0])
+			} else {
+				result[kv[0]] = ""
+			}
 		} else {
 			result[kv[0]] = kv[1]
 		}

commands/create.go

@@ -1,13 +1,16 @@
 package commands
 
 import (
+	"encoding/csv"
 	"fmt"
 	"os"
+	"strings"
 
 	"github.com/docker/buildx/driver"
 	"github.com/docker/buildx/store"
 	"github.com/docker/cli/cli"
 	"github.com/docker/cli/cli/command"
+	"github.com/google/shlex"
 	"github.com/moby/buildkit/util/appcontext"
 	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
@@ -22,6 +25,9 @@ type createOptions struct {
 	actionAppend bool
 	actionLeave  bool
 	use          bool
+	flags        string
+	configFile   string
+	driverOpts   []string
 	// upgrade      bool // perform upgrade of the driver
 }
 
@@ -107,6 +113,14 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
 		ng.Driver = driverName
 	}
 
+	var flags []string
+	if in.flags != "" {
+		flags, err = shlex.Split(in.flags)
+		if err != nil {
+			return errors.Wrap(err, "failed to parse buildkit flags")
+		}
+	}
+
 	var ep string
 	if in.actionLeave {
 		if err := ng.Leave(in.nodeName); err != nil {
@@ -128,7 +142,11 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
 				return err
 			}
 		}
-		if err := ng.Update(in.nodeName, ep, in.platform, len(args) > 0, in.actionAppend); err != nil {
+		m, err := csvToMap(in.driverOpts)
+		if err != nil {
+			return err
+		}
+		if err := ng.Update(in.nodeName, ep, in.platform, len(args) > 0, in.actionAppend, flags, in.configFile, m); err != nil {
 			return err
 		}
 	}
@@ -154,6 +172,11 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
 func createCmd(dockerCli command.Cli) *cobra.Command {
 	var options createOptions
 
+	var drivers []string
+	for s := range driver.GetFactories() {
+		drivers = append(drivers, s)
+	}
+
 	cmd := &cobra.Command{
 		Use:   "create [OPTIONS] [CONTEXT|ENDPOINT]",
 		Short: "Create a new builder instance",
@@ -166,9 +189,12 @@ func createCmd(dockerCli command.Cli) *cobra.Command {
 	flags := cmd.Flags()
 
 	flags.StringVar(&options.name, "name", "", "Builder instance name")
-	flags.StringVar(&options.driver, "driver", "", "Driver to use (eg. docker-container)")
+	flags.StringVar(&options.driver, "driver", "", fmt.Sprintf("Driver to use (available: %v)", drivers))
 	flags.StringVar(&options.nodeName, "node", "", "Create/modify node with given name")
+	flags.StringVar(&options.flags, "buildkitd-flags", "", "Flags for buildkitd daemon")
+	flags.StringVar(&options.configFile, "config", "", "BuildKit config file")
 	flags.StringArrayVar(&options.platform, "platform", []string{}, "Fixed platforms for current node")
+	flags.StringArrayVar(&options.driverOpts, "driver-opt", []string{}, "Options for the driver")
 
 	flags.BoolVar(&options.actionAppend, "append", false, "Append a node to builder instead of changing it")
 	flags.BoolVar(&options.actionLeave, "leave", false, "Remove a node from builder instead of changing it")
@@ -178,3 +204,22 @@ func createCmd(dockerCli command.Cli) *cobra.Command {
 
 	return cmd
 }
+
+func csvToMap(in []string) (map[string]string, error) {
+	m := make(map[string]string, len(in))
+	for _, s := range in {
+		csvReader := csv.NewReader(strings.NewReader(s))
+		fields, err := csvReader.Read()
+		if err != nil {
+			return nil, err
+		}
+		for _, v := range fields {
+			p := strings.SplitN(v, "=", 2)
+			if len(p) != 2 {
+				return nil, errors.Errorf("invalid value %q, expecting k=v", v)
+			}
+			m[p[0]] = p[1]
+		}
+	}
+	return m, nil
+}

commands/inspect.go

@@ -114,6 +114,9 @@ func runInspect(dockerCli command.Cli, in inspectOptions, args []string) error {
 				fmt.Fprintf(w, "Error:\t%s\n", err.Error())
 			} else {
 				fmt.Fprintf(w, "Status:\t%s\n", ngi.drivers[i].info.Status)
+				if len(n.Flags) > 0 {
+					fmt.Fprintf(w, "Flags:\t%s\n", strings.Join(n.Flags, " "))
+				}
 				fmt.Fprintf(w, "Platforms:\t%s\n", strings.Join(platformutil.Format(platformutil.Dedupe(append(n.Platforms, ngi.drivers[i].platforms...))), ", "))
 			}
 		}

commands/util.go

@@ -174,7 +174,7 @@ func driversForNodeGroup(ctx context.Context, dockerCli command.Cli, ng *store.N
 				// TODO: replace the following line with dockerclient.WithAPIVersionNegotiation option in clientForEndpoint
 				dockerapi.NegotiateAPIVersion(ctx)
 
-				d, err := driver.GetDriver(ctx, "buildx_buildkit_"+n.Name, f, dockerapi)
+				d, err := driver.GetDriver(ctx, "buildx_buildkit_"+n.Name, f, dockerapi, n.Flags, n.ConfigFile, n.DriverOpts)
 				if err != nil {
 					di.Err = err
 					return nil
@@ -251,7 +251,7 @@ func getDefaultDrivers(ctx context.Context, dockerCli command.Cli) ([]build.Driv
 		return driversForNodeGroup(ctx, dockerCli, ng)
 	}
 
-	d, err := driver.GetDriver(ctx, "buildx_buildkit_default", nil, dockerCli.Client())
+	d, err := driver.GetDriver(ctx, "buildx_buildkit_default", nil, dockerCli.Client(), nil, "", nil)
 	if err != nil {
 		return nil, err
 	}

driver/docker-container/driver.go

@@ -1,6 +1,8 @@
 package docker
 
 import (
+	"archive/tar"
+	"bytes"
 	"context"
 	"io"
 	"io/ioutil"
@@ -20,11 +22,13 @@ import (
 	"github.com/pkg/errors"
 )
 
-var buildkitImage = "moby/buildkit:master" // TODO: make this verified and configuratble
+var defaultBuildkitImage = "moby/buildkit:buildx-stable-1" // TODO: make this verified
 
 type Driver struct {
 	driver.InitConfig
 	factory driver.Factory
+	netMode string
+	image   string
 }
 
 func (d *Driver) Bootstrap(ctx context.Context, l progress.Logger) error {
@@ -49,8 +53,12 @@ func (d *Driver) Bootstrap(ctx context.Context, l progress.Logger) error {
 }
 
 func (d *Driver) create(ctx context.Context, l progress.SubLogger) error {
-	if err := l.Wrap("pulling image "+buildkitImage, func() error {
+	imageName := defaultBuildkitImage
-		rc, err := d.DockerAPI.ImageCreate(ctx, buildkitImage, types.ImageCreateOptions{})
+	if d.image != "" {
+		imageName = d.image
+	}
+	if err := l.Wrap("pulling image "+imageName, func() error {
+		rc, err := d.DockerAPI.ImageCreate(ctx, imageName, types.ImageCreateOptions{})
 		if err != nil {
 			return err
 		}
@@ -59,15 +67,34 @@ func (d *Driver) create(ctx context.Context, l progress.SubLogger) error {
 	}); err != nil {
 		return err
 	}
+
+	cfg := &container.Config{
+		Image: imageName,
+	}
+	if d.InitConfig.BuildkitFlags != nil {
+		cfg.Cmd = d.InitConfig.BuildkitFlags
+	}
+
 	if err := l.Wrap("creating container "+d.Name, func() error {
-		_, err := d.DockerAPI.ContainerCreate(ctx, &container.Config{
+		hc := &container.HostConfig{
-			Image: buildkitImage,
-		}, &container.HostConfig{
 			Privileged: true,
-		}, &network.NetworkingConfig{}, d.Name)
+		}
+		if d.netMode != "" {
+			hc.NetworkMode = container.NetworkMode(d.netMode)
+		}
+		_, err := d.DockerAPI.ContainerCreate(ctx, cfg, hc, &network.NetworkingConfig{}, d.Name)
 		if err != nil {
 			return err
 		}
+		if f := d.InitConfig.ConfigFile; f != "" {
+			buf, err := readFileToTar(f)
+			if err != nil {
+				return err
+			}
+			if err := d.DockerAPI.CopyToContainer(ctx, d.Name, "/", buf, dockertypes.CopyToContainerOptions{}); err != nil {
+				return err
+			}
+		}
 		if err := d.start(ctx, l); err != nil {
 			return err
 		}
@@ -239,3 +266,26 @@ type demux struct {
 func (d *demux) Read(dt []byte) (int, error) {
 	return d.Reader.Read(dt)
 }
+
+func readFileToTar(fn string) (*bytes.Buffer, error) {
+	buf := bytes.NewBuffer(nil)
+	tw := tar.NewWriter(buf)
+	dt, err := ioutil.ReadFile(fn)
+	if err != nil {
+		return nil, err
+	}
+	if err := tw.WriteHeader(&tar.Header{
+		Name: "/etc/buildkit/buildkitd.toml",
+		Size: int64(len(dt)),
+		Mode: 0644,
+	}); err != nil {
+		return nil, err
+	}
+	if _, err := tw.Write(dt); err != nil {
+		return nil, err
+	}
+	if err := tw.Close(); err != nil {
+		return nil, err
+	}
+	return buf, nil
+}

driver/docker-container/factory.go

@@ -37,8 +37,22 @@ func (f *factory) New(ctx context.Context, cfg driver.InitConfig) (driver.Driver
 	if cfg.DockerAPI == nil {
 		return nil, errors.Errorf("%s driver requires docker API access", f.Name())
 	}
+	d := &Driver{factory: f, InitConfig: cfg}
+	for k, v := range cfg.DriverOpts {
+		switch k {
+		case "network":
+			d.netMode = v
+			if v == "host" {
+				d.InitConfig.BuildkitFlags = append(d.InitConfig.BuildkitFlags, "--allow-insecure-entitlement=network.host")
+			}
+		case "image":
+			d.image = v
+		default:
+			return nil, errors.Errorf("invalid driver option %s for docker-container driver", k)
+		}
+	}
 
-	return &Driver{factory: f, InitConfig: cfg}, nil
+	return d, nil
 }
 
 func (f *factory) AllowsInstances() bool {

driver/docker/factory.go

@@ -44,6 +44,9 @@ func (f *factory) New(ctx context.Context, cfg driver.InitConfig) (driver.Driver
 	if cfg.DockerAPI == nil {
 		return nil, errors.Errorf("docker driver requires docker API access")
 	}
+	if cfg.ConfigFile != "" {
+		return nil, errors.Errorf("setting config file is not supported for docker driver, use dockerd configuration file")
+	}
 
 	return &Driver{factory: f, InitConfig: cfg}, nil
 }

driver/manager.go

@@ -23,10 +23,11 @@ type BuildkitConfig struct {
 
 type InitConfig struct {
 	// This object needs updates to be generic for different drivers
-	Name           string
+	Name          string
-	DockerAPI      dockerclient.APIClient
+	DockerAPI     dockerclient.APIClient
-	BuildkitConfig BuildkitConfig
+	BuildkitFlags []string
-	Meta           map[string]interface{}
+	ConfigFile    string
+	DriverOpts    map[string]string
 }
 
 var drivers map[string]Factory
@@ -71,10 +72,13 @@ func GetFactory(name string, instanceRequired bool) Factory {
 	return nil
 }
 
-func GetDriver(ctx context.Context, name string, f Factory, api dockerclient.APIClient) (Driver, error) {
+func GetDriver(ctx context.Context, name string, f Factory, api dockerclient.APIClient, flags []string, config string, do map[string]string) (Driver, error) {
 	ic := InitConfig{
-		DockerAPI: api,
+		DockerAPI:     api,
-		Name:      name,
+		Name:          name,
+		BuildkitFlags: flags,
+		ConfigFile:    config,
+		DriverOpts:    do,
 	}
 	if f == nil {
 		var err error
@@ -85,3 +89,7 @@ func GetDriver(ctx context.Context, name string, f Factory, api dockerclient.API
 	}
 	return f.New(ctx, ic)
 }
+
+func GetFactories() map[string]Factory {
+	return drivers
+}

go.mod

@@ -35,6 +35,7 @@ require (
 	github.com/gogo/protobuf v1.2.1 // indirect
 	github.com/google/certificate-transparency-go v1.0.21 // indirect
 	github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf // indirect
+	github.com/google/shlex v0.0.0-20150127133951-6f45313302b9
 	github.com/gorilla/mux v1.7.0 // indirect
 	github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed // indirect
 	github.com/hashicorp/go-version v1.1.0 // indirect

store/nodegroup.go

@@ -16,9 +16,12 @@ type NodeGroup struct {
 }
 
 type Node struct {
-	Name      string
+	Name       string
-	Endpoint  string
+	Endpoint   string
-	Platforms []specs.Platform
+	Platforms  []specs.Platform
+	Flags      []string
+	ConfigFile string
+	DriverOpts map[string]string
 }
 
 func (ng *NodeGroup) Leave(name string) error {
@@ -33,7 +36,7 @@ func (ng *NodeGroup) Leave(name string) error {
 	return nil
 }
 
-func (ng *NodeGroup) Update(name, endpoint string, platforms []string, endpointsSet bool, actionAppend bool) error {
+func (ng *NodeGroup) Update(name, endpoint string, platforms []string, endpointsSet bool, actionAppend bool, flags []string, configFile string, do map[string]string) error {
 	i := ng.findNode(name)
 	if i == -1 && !actionAppend {
 		if len(ng.Nodes) > 0 {
@@ -55,6 +58,9 @@ func (ng *NodeGroup) Update(name, endpoint string, platforms []string, endpoints
 		if len(platforms) > 0 {
 			n.Platforms = pp
 		}
+		if flags != nil {
+			n.Flags = flags
+		}
 		ng.Nodes[i] = n
 		if err := ng.validateDuplicates(endpoint, i); err != nil {
 			return err
@@ -72,9 +78,12 @@ func (ng *NodeGroup) Update(name, endpoint string, platforms []string, endpoints
 	}
 
 	n := Node{
-		Name:      name,
+		Name:       name,
-		Endpoint:  endpoint,
+		Endpoint:   endpoint,
-		Platforms: pp,
+		Platforms:  pp,
+		ConfigFile: configFile,
+		Flags:      flags,
+		DriverOpts: do,
 	}
 	ng.Nodes = append(ng.Nodes, n)
 

store/nodegroup_test.go

@@ -11,16 +11,16 @@ func TestNodeGroupUpdate(t *testing.T) {
 	t.Parallel()
 
 	ng := &NodeGroup{}
-	err := ng.Update("foo", "foo0", []string{"linux/amd64"}, true, false)
+	err := ng.Update("foo", "foo0", []string{"linux/amd64"}, true, false, []string{"--debug"}, "", nil)
 	require.NoError(t, err)
 
-	err = ng.Update("foo1", "foo1", []string{"linux/arm64", "linux/arm/v7"}, true, true)
+	err = ng.Update("foo1", "foo1", []string{"linux/arm64", "linux/arm/v7"}, true, true, nil, "", nil)
 	require.NoError(t, err)
 
 	require.Equal(t, len(ng.Nodes), 2)
 
 	// update
-	err = ng.Update("foo", "foo2", []string{"linux/amd64", "linux/arm"}, true, false)
+	err = ng.Update("foo", "foo2", []string{"linux/amd64", "linux/arm"}, true, false, nil, "", nil)
 	require.NoError(t, err)
 
 	require.Equal(t, len(ng.Nodes), 2)
@@ -28,9 +28,11 @@ func TestNodeGroupUpdate(t *testing.T) {
 	require.Equal(t, []string{"linux/arm64"}, platformutil.Format(ng.Nodes[1].Platforms))
 
 	require.Equal(t, "foo2", ng.Nodes[0].Endpoint)
+	require.Equal(t, []string{"--debug"}, ng.Nodes[0].Flags)
+	require.Equal(t, []string(nil), ng.Nodes[1].Flags)
 
 	// duplicate endpoint
-	err = ng.Update("foo1", "foo2", nil, true, false)
+	err = ng.Update("foo1", "foo2", nil, true, false, nil, "", nil)
 	require.Error(t, err)
 	require.Contains(t, err.Error(), "duplicate endpoint")
 

vendor/modules.txt

@@ -36,6 +36,8 @@ github.com/containerd/containerd/remotes
 github.com/containerd/containerd/remotes/docker
 github.com/containerd/containerd/log
 github.com/containerd/containerd/content/local
+github.com/containerd/containerd/containers
+github.com/containerd/containerd/oci
 github.com/containerd/containerd/labels
 github.com/containerd/containerd/reference
 github.com/containerd/containerd/version
@@ -44,12 +46,10 @@ github.com/containerd/containerd/sys
 github.com/containerd/containerd/api/services/content/v1
 github.com/containerd/containerd/content/proxy
 github.com/containerd/containerd/services/content/contentserver
-github.com/containerd/containerd/containers
-github.com/containerd/containerd/oci
-github.com/containerd/containerd
-github.com/containerd/containerd/namespaces
 github.com/containerd/containerd/mount
+github.com/containerd/containerd/namespaces
 github.com/containerd/containerd/snapshots
+github.com/containerd/containerd
 github.com/containerd/containerd/api/services/containers/v1
 github.com/containerd/containerd/api/services/diff/v1
 github.com/containerd/containerd/api/services/events/v1
@@ -83,12 +83,12 @@ github.com/containerd/containerd/events/exchange
 github.com/containerd/containerd/identifiers
 # github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc
 github.com/containerd/continuity
+github.com/containerd/continuity/fs
 github.com/containerd/continuity/pathdriver
 github.com/containerd/continuity/devices
 github.com/containerd/continuity/driver
 github.com/containerd/continuity/proto
 github.com/containerd/continuity/sysx
-github.com/containerd/continuity/fs
 github.com/containerd/continuity/syscallx
 # github.com/containerd/fifo v0.0.0-20190226154929-a9fb20d87448
 github.com/containerd/fifo
@@ -266,6 +266,7 @@ github.com/moby/buildkit/session
 github.com/moby/buildkit/session/secrets/secretsprovider
 github.com/moby/buildkit/session/sshforward/sshprovider
 github.com/moby/buildkit/session/upload/uploadprovider
+github.com/moby/buildkit/util/entitlements
 github.com/moby/buildkit/util/appcontext
 github.com/moby/buildkit/identity
 github.com/moby/buildkit/util/progress/progressui
@@ -285,7 +286,6 @@ github.com/moby/buildkit/session/grpchijack
 github.com/moby/buildkit/solver/pb
 github.com/moby/buildkit/util/apicaps
 github.com/moby/buildkit/util/appdefaults
-github.com/moby/buildkit/util/entitlements
 github.com/moby/buildkit/session/secrets
 github.com/moby/buildkit/session/sshforward
 github.com/moby/buildkit/session/upload