--output type=docker,tar=true and tar=false create different files

[SpecLad]

Take this Dockerfile:

FROM scratch

Build it in two ways:

# buildctl build --output type=docker,dest=/tmp/a,tar=false --frontend dockerfile.v0 --local context=. --local dockerfile=.
[+] Building 0.1s (3/3) FINISHED
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 50B                                                                                0.0s
 => [internal] load .dockerignore                                                                                  0.0s
 => => transferring context: 2B                                                                                    0.0s
 => exporting to docker image format                                                                               0.0s
 => => exporting layers                                                                                            0.0s
 => => exporting manifest sha256:3a58ad119f0dae830e39299dd9d94100e3a764665fba8810f9a873d3ac716aec                  0.0s
 => => exporting config sha256:471a1b8817eefb6569017c1a76f288e0d4e5c8476eb199485c469d0b033168bf                    0.0s
# buildctl build --output type=docker,dest=/tmp/a.tar,tar=true --frontend dockerfile.v0 --local context=. --local dockerfile=.
[+] Building 0.1s (3/3) FINISHED
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 50B                                                                                0.0s
 => [internal] load .dockerignore                                                                                  0.0s
 => => transferring context: 2B                                                                                    0.0s
 => exporting to docker image format                                                                               0.0s
 => => exporting layers                                                                                            0.0s
 => => exporting manifest sha256:3a58ad119f0dae830e39299dd9d94100e3a764665fba8810f9a873d3ac716aec                  0.0s
 => => exporting config sha256:471a1b8817eefb6569017c1a76f288e0d4e5c8476eb199485c469d0b033168bf                    0.0s
 => => sending tarball 

Examine the results:

$ tar -tf /tmp/a.tar
blobs/
blobs/sha256/
blobs/sha256/3a58ad119f0dae830e39299dd9d94100e3a764665fba8810f9a873d3ac716aec
blobs/sha256/471a1b8817eefb6569017c1a76f288e0d4e5c8476eb199485c469d0b033168bf
index.json
manifest.json
oci-layout
$ find /tmp/a
/tmp/a
/tmp/a/ingest
/tmp/a/index.json
/tmp/a/oci-layout
/tmp/a/blobs
/tmp/a/blobs/sha256
/tmp/a/blobs/sha256/471a1b8817eefb6569017c1a76f288e0d4e5c8476eb199485c469d0b033168bf
/tmp/a/blobs/sha256/3a58ad119f0dae830e39299dd9d94100e3a764665fba8810f9a873d3ac716aec

The tar=true output has the manifest.json file, while the tar=false output has the ingest directory. From the description of the option I would expect both outputs to be the same (except for one of them being an archive, of course).

Version information:

# buildctl --version
buildctl github.com/moby/buildkit v0.18.0 95d190ef4f18b57c717eaad703b67cb2be781ebb

[jedevc]

There's some context for this behavior in #1949 (comment).

I think what you're seeing is expected, but will let @tonistiigi confirm.

[SpecLad]

There's some context for this behavior in #1949 (comment).

I looked at that comment (and issue), but I'm not sure how that's applicable here. That issue is about the oci exporter; this one is about docker.

[tonistiigi]

The tar=true output has the manifest.json file, while the tar=false output has the ingest directory. From the description of the option I would expect both outputs to be the same (except for one of them being an archive, of course).

manifest.json is file providing compatibility with docker load on older daemons. It is not part of the OCI definition.

ingest directory is side-effect by containerd library and should be fixed after containerd/containerd#10894

[SpecLad]

manifest.json is file providing compatibility with docker load on older daemons. It is not part of the OCI definition.

Sure, I get that. But why is it only added when tar=true?

[tonistiigi]

Sure, I get that. But why is it only added when tar=true?

Because docker load can only load from a tar file. While OCI layout itself is a directory structure and for example build contexts from oci-layout take directories as path.

[tonistiigi]

We might just make --output type=docker,dest=/tmp/a.tar,tar=false invalid. tar was meant to be option for the oci exporter and I think that it is somewhat working for docker exporter as well is accidental. Don't want to break someone setting it for no reason though.

[SpecLad]

We might just make --output type=docker,dest=/tmp/a.tar,tar=true invalid.

I assume you mean tar=false?

Well, if you want to disallow this, it's certainly your prerogative. But let me just explain my use case here.

I wanted to build an image and simultaneously load it to Docker and save it to the filesystem as an unpacked OCI image, to be used as a build context for another build. My first attempt didn't work because of #5556. So I figured I would try to save the image in the Docker format instead (which I assume would also work as a build context), and then tar it up manually and pipe it to docker image load. But that doesn't work either, because there's no manifest.json in the resulting image. So that's why it would be nice if manifest.json would be saved regardless of the value of tar.

[tonistiigi]

I assume you mean tar=false?

Yes, sorry.

I wanted to build an image and simultaneously load it to Docker and save it to the filesystem as an unpacked OCI image, to be used as a build context for another build.

#5556 looks like a bug if it works correctly with one output but not with both together.