`go-1.19.3` tag starts complaining about missing go.mod file

[m90]

I've been using xgo for a while now to build cross-platform binaries for my application. Currently, I'm on techknowlogick/xgo:go-1.19.x.

Starting today, builds would start failing with the following error message:

 > [compiler 10/12] RUN chmod +x ./compile.sh && ./compile.sh && rm ./compile.sh:
#77 0.388 Cross compiling github.com/offen/offen/server/cmd/offen...
#77 0.431 Go modules are enabled but go.mod was not found in the source folder.
#77 0.431 2022/11/24 10:03:14 Failed to cross compile package: exit status 10.

The command from compile.sh looks like this:

  xgo \
    --targets=$TARGETS \
    --tags 'osusergo netgo static_build sqlite_omit_load_extension' \
    --ldflags="-linkmode external -extldflags '$LDFLAGS' -s -w -X github.com/offen/offen/server/config.Revision=$GIT_REVISION" \
    github.com/offen/offen/server/cmd/offen

When this error is raised, there is a go.mod file in github.com/offen/offen/server/go.mod.

I assume this behavior was introduced in the recent go-1.19.3 release as downgrading to go-1.19.2 makes the build pass again.

Am I doing something wrong when building my binaries or does go-1.19.3 contain a regression somehow?

This is how assets are copied into building image:

FROM techknowlogick/xgo:go-1.19.x as compiler

ARG rev
ENV GIT_REVISION=$rev
ARG targets
ENV TARGETS=$targets

COPY ./server /go/src/github.com/offen/offen/server

ENV GOPATH /go
WORKDIR /build

COPY ./build/compile.sh .
RUN chmod +x ./compile.sh && ./compile.sh && rm ./compile.sh

[techknowlogick]

This could be related to dc59b5d which changes the GO111MODULES detection behaviour.

[kolaente]

I've noticed that as well: https://drone.kolaente.de/vikunja/api/4498/2/6

[kolaente]

I think the problem is caused by this line here:

xgo/docker/base/build.sh

Line 170 in 63bb785

NAME=$(sed -n 's/module\ \(.*\)/\1/p' /source/go.mod)

This one was changed during #180 but only formatted differently. This leaves two conclusions:

1. Either the if surrounding the sed call was not executed at all before
2. Or the call always failed, the rest of the script just ignored it and proceeded anyway

I think the path should not be hardcoded. This might work when running xgo via the shell but once running in CI or similar where xgo is called directly that path might be different. Shouldn't the current working directory work as well?

[techknowlogick]

@kolaente I wonder why it fails for you, as you have a similar build process to Gitea.

[kolaente]

@techknowlogick is gitea using the latest image?

[techknowlogick]

Gitea is using :go-1.19.x (which is the same as :latest)

[6543]

Ah ... this use a absolite path ... woodpecker also had to stick to an older version

[6543]

https://github.com/woodpecker-ci/woodpecker/pull/1499/files

[techknowlogick]

Ah yeah, it's working for Gitea as the workspace is hardcoded as /source https://github.com/go-gitea/gitea/blob/main/.drone.yml#L685-L687 so go.mod is copied in

[6543]

well I dont think there should be a hardcoded workspace asumed :/

[zeripath]

So looking at xgo.go if it detects that there is a go.mod present in the root of the repository it will automount the repository as /source

xgo/xgo.go

Lines 270 to 273 in a9c7973

	if _, err := os.Stat(config.Repository + "/go.mod"); err == nil {
	usesModules = true
	}
	if !usesModules {

xgo/xgo.go

Lines 386 to 406 in a9c7973

	if usesModules {
	args = append(args, []string{"-e", "GO111MODULE=on"}...)
	args = append(args, []string{"-v", build.Default.GOPATH + ":/go"}...)
	// Map this repository to the /source folder
	absRepository, err := filepath.Abs(config.Repository)
	if err != nil {
	log.Fatalf("Failed to locate requested module repository: %v.", err)
	}
	args = append(args, []string{"-v", absRepository + ":/source"}...)
	fmt.Printf("Enabled Go module support\n")
	// Check whether it has a vendor folder, and if so, use it
	vendorPath := absRepository + "/vendor"
	vendorfolder, err := os.Stat(vendorPath)
	if !os.IsNotExist(err) && vendorfolder.Mode().IsDir() {
	args = append(args, []string{"-e", "FLAG_MOD=vendor"}...)
	fmt.Printf("Using vendored Go module dependencies\n")
	}
	} else {

Do we have an example of the repo in question?

---

There's essentially a conflict with the way that xgo.go, go and the docker build.sh are detecting go modules.

I guess this might need to be rethought a bit. Perhaps the source-repo should just be mounted at /source always? Or the decision about where the GOPATH is should be handled in the base.sh?

[zeripath]

OK, so re-reading the initial comment - the problem here is on Line 269:

xgo/xgo.go

Lines 269 to 273 in a9c7973

	if strings.HasPrefix(config.Repository, string(filepath.Separator)) || strings.HasPrefix(config.Repository, ".") {
	if _, err := os.Stat(config.Repository + "/go.mod"); err == nil {
	usesModules = true
	}
	if !usesModules {

config.Repository is flag.Args()[0] i.e. "github.com/offen/offen/server/cmd/offen"

Thus useModules remains false as it is never set and so the /source mount point is never run.

Then the changes in dc59b5d apply and overrule GOMODULES and force the use of /source.

---

So there are two bugs:

• Either dc59b5d is completely wrong and we should not attempt to determine GOMODULES on a per go version basis - or we should always mount the current source as /source <- I think this second option is probably the correct one.
  • I'm confused as to how this was ever building in the first place - @m90 are you building within the GOPATH? That's the only way I can see that the source code was being passed to the docker build...
• We should be able to cope with relative paths in the config.Repository.

(As an aside, I think there's another issue with $NAME determination in build.sh )

---

EDIT:

AGH! You're not running this using a relative path but a go get url...

OK so we need to change build.sh to handle that!

[m90]

I'm confused as to how this was ever building in the first place - @m90 are you building within the GOPATH? That's the only way I can see that the source code was being passed to the docker build...

Yes. I am. GOPATH is at /go and source is at /go/src/github.com/offen/offen/server