x/tools/gopls: dependency leaks across modules

[fhs]

What version of Go are you using (go version)?

$ go version
go version devel +ce656af9b5 Sun Jun 2 19:28:21 2019 +0000 linux/amd64
$ gopls version
golang.org/x/tools/cmd/gopls
    golang.org/x/tools@v0.0.0-20190608022120-eacb66d2a7c3 h1:sU3tSV6wDhWsvf9NjL0FzRjgAmYnQL5NEhdmcN16UEg=

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/fhs/.cache/go-build"
GOENV="/home/fhs/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/fhs/go"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/big/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/big/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build362340636=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Load more than one module in gopls. Each module has its own workspace folder, passed in the workspaceFolders initialization option.

What did you expect to see?

No changes to go.mod file for any of the modules.

What did you see instead?

I have 2 files open in my editor, where the two files are from two different modules. When I start the LSP client, it sends a DidOpen notification for those two files. Depending on which notification was sent first, the dependencies of one module gets added to the go.mod file of the other module. Example:

DidOpen file:///path/to/mod1/foo.go
DidOpen file:///path/to/mod2/foo.go
# mod2's go.mod file will get mod1 dependencies added

Similarly, for 3 modules:

DidOpen file:///path/to/mod1/foo.go
DidOpen file:///path/to/mod2/foo.go
DidOpen file:///path/to/mod3/foo.go
# mod2's go.mod file will get mod1 dependencies added
# mod3's go.mod file will get mod1 and mod2 dependencies added

[stamblerre]

I am not able to reproduce this in VSCode. What LSP client are you using?

[fhs]

I'm using acme-lsp. I'm also unsuccessful at reproducing it in VSCode.

With latest gopls from master branch, it seems I need to do few go-to-defenitions before I can reproduce the issue. Here is my gopls trace output: gopls.log. I'm not sure what exactly I'm doing different compared to VSCode. If you spot anything from the log it'll be helpful.

[stamblerre]

Based on the fact that this can't be reproduced in VSCode, I'm inclined to believe that it could be a bug with the Acme client.

The only thing that can edit the go.mod files in gopls would be the calls to go/packages.Load or import organization. It seems like you are not using the import organization feature, based on your logs, so my guess is that it would be go/packages. The output of go/packages is logged here. If you would like to investigate this further, perhaps you could determine exactly which call to packages.Load causes the change. You could also log additional information, like the contents of the packages.Config.

[itchyny]

I believe this is caused by go list command in determineRootDirs, where PWD seems not to be correct. I still cannot find the root cause, but the PWD of go list on didOpen in new project is in the root directory of the previous project. I'm debugging the bug with fatih/vim-go#2383 (comment).

debug log

sent: Content-Length: 544
�
�{"method":"initialize","jsonrpc":"2.0","id":1,"params":{"rootUri":"file:///private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a","capabilities":{"workspace":{"workspaceFolders":true,"configuration":true,"didChangeConfiguration":{"dynamicRegistration":true}},"textDocument":{"hover":{"contentFormat":["plaintext"]}}},"processId":89698,"workspaceFolders":[{"uri":"file:///private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a","name":"/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a"}]}}
received: Content-Length: 292
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"71.561897ms for GOROOT= GOPATH=/Users/itchyny/.go GO111MODULE=on PWD=/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a go "list" "-m" "-json" "all", stderr: \u003c\u003c\u003e\u003e\n"}}
received: Content-Length: 386
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"74.188355ms for GOROOT= GOPATH=/Users/itchyny/.go GO111MODULE=on PWD=/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a go "list" "-e" "-json" "-compiled=true" "-test=true" "-export=false" "-deps=true" "-find=false" "--" "builtin", stderr: \u003c\u003c\u003e\u003e\n"}}
received: Content-Length: 616
�
�{"jsonrpc":"2.0","result":{"capabilities":{"textDocumentSync":{"openClose":true,"change":2,"save":{}},"hoverProvider":true,"completionProvider":{"triggerCharacters":["."]},"signatureHelpProvider":{"triggerCharacters":["(",","]},"definitionProvider":true,"referencesProvider":true,"documentHighlightProvider":true,"documentSymbolProvider":true,"codeActionProvider":true,"documentFormattingProvider":true,"renameProvider":true,"documentLinkProvider":{},"typeDefinitionProvider":true,"workspace":{"workspaceFolders":{"supported":true,"changeNotifications":"workspace/didChangeWorkspaceFolders"}}},"custom":null},"id":1}
sent: Content-Length: 52
�
�{"method":"initialized","jsonrpc":"2.0","params":{}}
sent: Content-Length: 290
�
�{"method":"textDocument/didOpen","jsonrpc":"2.0","params":{"textDocument":{"uri":"file:///private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a/main.go","languageId":"go","text":"package main\n\nimport "golang.org/x/xerrors"\n\nfunc main() {\n xerrors.New("")\n}\n"}}}
received: Content-Length: 268
�
�{"jsonrpc":"2.0","method":"client/registerCapability","params":{"registrations":[{"id":"workspace/didChangeConfiguration","method":"workspace/didChangeConfiguration"},{"id":"workspace/didChangeWorkspaceFolders","method":"workspace/didChangeWorkspaceFolders"}]},"id":1}
sent: Content-Length: 38
�
�{"id":1,"jsonrpc":"2.0","result":null}
stderr: 2019/08/10 08:31:49 Debug serving
received: Content-Length: 370
�
�{"jsonrpc":"2.0","method":"workspace/configuration","params":{"items":[{"scopeUri":"file:///private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a","section":"gopls"},{"scopeUri":"file:///private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a","section":"/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a"}]},"id":2}
sent: Content-Length: 131
�
�{"id":2,"jsonrpc":"2.0","result":[{"buildFlags":[],"hoverKind":"NoDocumentation"},{"buildFlags":[],"hoverKind":"NoDocumentation"}]}
stderr: Port = 62655
received: Content-Length: 1278
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"Build info\n----------\ngolang.org/x/tools/cmd/gopls v0.1.3-cmd.gopls\n golang.org/x/tools@(devel)\n golang.org/x/sync@v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=\n golang.org/x/xerrors@v0.0.0-20190717185122-a985d3407aa7 h1:9zdDQZ7Thm29KFXgAX/+yaf3eVbP7djjWp/dXAppNCc=\n\nGo info\n-------\ngo version go1.12.7 darwin/amd64\n\nGOARCH="amd64"\nGOBIN=""\nGOCACHE="/Users/itchyny/Library/Caches/go-build"\nGOEXE=""\nGOFLAGS=""\nGOHOSTARCH="amd64"\nGOHOSTOS="darwin"\nGOOS="darwin"\nGOPATH="/Users/itchyny/.go"\nGOPROXY=""\nGORACE=""\nGOROOT="/usr/local/Cellar/go/1.12.7/libexec"\nGOTMPDIR=""\nGOTOOLDIR="/usr/local/Cellar/go/1.12.7/libexec/pkg/tool/darwin_amd64"\nGCCGO="gccgo"\nCC="clang"\nCXX="clang++"\nCGO_ENABLED="1"\nGOMOD="/dev/null"\nCGO_CFLAGS="-g -O2"\nCGO_CPPFLAGS=""\nCGO_CXXFLAGS="-g -O2"\nCGO_FFLAGS="-g -O2"\nCGO_LDFLAGS="-g -O2"\nPKG_CONFIG="pkg-config"\nGOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/go-build843905483=/tmp/go-build -gno-record-gcc-switches -fno-common"\n"}}
received: Content-Length: 292
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"37.503774ms for GOROOT= GOPATH=/Users/itchyny/.go GO111MODULE=on PWD=/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a go "list" "-m" "-json" "all", stderr: \u003c\u003c\u003e\u003e\n"}}
received: Content-Length: 451
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"228.806934ms for GOROOT= GOPATH=/Users/itchyny/.go GO111MODULE=on PWD=/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a go "list" "-e" "-json" "-compiled=true" "-test=true" "-export=false" "-deps=true" "-find=false" "--" "/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a", stderr: \u003c\u003c\u003e\u003e\n"}}
received: Content-Length: 203
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"go/packages.Load\n\tpackage = a\n\tfiles = [/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a/main.go]"}}
received: Content-Length: 111
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"go/packages.Load\n\tpackages = 1"}}
received: Content-Length: 183
�
�{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":"file:///private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a/main.go","diagnostics":[]}}

sent: Content-Length: 309
�
�{"method":"textDocument/didOpen","jsonrpc":"2.0","params":{"textDocument":{"uri":"file:///private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/b/main.go","languageId":"go","text":"package main\n\nimport "golang.org/x/sys/unix"\n\nfunc main() {\n unix.SysctlRaw("kern.boottime")\n}\n\n"}}}
received: Content-Length: 293
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"107.029404ms for GOROOT= GOPATH=/Users/itchyny/.go GO111MODULE=on PWD=/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a go "list" "-m" "-json" "all", stderr: \u003c\u003c\u003e\u003e\n"}}
received: Content-Length: 495
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"64.85989ms for GOROOT= GOPATH=/Users/itchyny/.go GO111MODULE=on PWD=/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a go "list" "-e" "-json" "-compiled=true" "-test=true" "-export=false" "-deps=true" "-find=false" "--" "/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/b", stderr: \u003c\u003cgo: directory ../b outside available modules\n\u003e\u003e\n"}}
received: Content-Length: 538
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"1.678329771s for GOROOT= GOPATH=/Users/itchyny/.go GO111MODULE=on PWD=/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/a go "list" "-e" "-json" "-compiled=true" "-test=true" "-export=false" "-deps=true" "-find=false" "--" "/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/b/main.go", stderr: \u003c\u003cgo: finding golang.org/x/sys/unix latest\ngo: finding golang.org/x/sys latest\n\u003e\u003e\n"}}
received: Content-Length: 224
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"go/packages.Load\n\tpackage = command-line-arguments\n\tfiles = [/private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/b/main.go]"}}
received: Content-Length: 111
�
�{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"go/packages.Load\n\tpackages = 1"}}
received: Content-Length: 183
�
�{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":"file:///private/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/tmp.FiVYTGf3/b/main.go","diagnostics":[]}}

[fhs]

I've added logging of packages.Config. I noticed that it's using Overlay in acme but not in VSCode. In particular, this one looks suspecious:

[Trace - 8:22:19 PM] Received notification 'window/logMessage' in 0ms.
Params: {"type":3,"message":"go/packages.Load\n\tconfig = \u0026packages.Config{Mode:543, Context:context.Context(nil), Logf:(func(string, ...interface {}))(0x971600), 
Dir:\"/home/fhs/go/src/github.com/fhs/acme-lsp\", Env:[]string{...}, BuildFlags:[]string(nil), Fset:(*token.FileSet)(0xc000195a40), ParseFile:(func(*token.FileSet, string, []uint8) (*ast.File, error))(0x9715b0), Tests:true, 
Overlay:map[string][]uint8{
\"/home/fhs/go/src/github.com/fhs/acme-lsp/cmd/acme-lsp/main.go\":[]uint8{...}, 
\"/home/fhs/go/src/github.com/rjkroege/edwood/acme.go\":[]uint8{...}}}"}

Can Overlay have files from two different modules?

Later on, this go list command modifies the go.mod file (adds edwood as an indirect dependency to acme-lsp):

[Trace - 8:22:19 PM] Received notification 'window/logMessage' in 0ms.
Params: {"type":3,"message":"512.764166ms for GOROOT= GOPATH=/home/fhs/go GO111MODULE=on PWD=/home/fhs/go/src/github.com/fhs/acme-lsp go \"list\" \"-e\" \"-json\" \"-compiled=true\" \"-test=true\" \"-export=false\" \"-deps=true\" \"-find=false\" \"--\" \"net/http/pprof\" \"github.com/rjkroege/edwood/internal/draw\" \"github.com/rjkroege/edwood/internal/dumpfile\" \"github.com/rjkroege/edwood/internal/frame\" \"net/http\" \"os/signal\", stderr: \u003c\u003c\u003e\u003e\n"}

[stamblerre]

Thank you for digging into this further! Overlays can have files from different modules, and the extra go list calls for overlays are likely responsible here. The solution to this will probably be making sure that overlays are built per-view and not per-session here.