x/tools/gopls: missing semantic tokens for function parameters in block

[danicc097]

gopls version

Build info

golang.org/x/tools/gopls v0.9.5
golang.org/x/tools/[email protected] h1:F3TO5th6TUg40FHrpxZfMkMlVaaTmByeEvylgVHGH2A=
github.com/BurntSushi/[email protected] h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
github.com/google/[email protected] h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
github.com/sergi/[email protected] h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
golang.org/x/[email protected] h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
golang.org/x/exp/[email protected] h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
golang.org/x/[email protected] h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
golang.org/x/[email protected] h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
golang.org/x/[email protected] h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
golang.org/x/[email protected] h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/[email protected] h1:glzimF7qHZuKVEiMbE7UqBu44MyTjt5u6j3Jz+rfMRM=
golang.org/x/[email protected] h1:YnB27EXBD8XxB0JcaOeluuvhF2kS4DrH0k+lpopG2xc=
honnef.co/go/[email protected] h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34=
mvdan.cc/[email protected] h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
mvdan.cc/xurls/[email protected] h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
go: go1.19.1

go env

GO111MODULE=""
GOARCH="amd64"
GOBIN="//bin/tools"
GOCACHE="/home/daniel/.cache/go-build"
GOENV="/home/daniel/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/daniel/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/daniel/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="//go.mod"
GOWORK="/***/go.work"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2391207226=/tmp/go-build -gno-record-gcc-switches"

What did you do?

What did you expect to see?

What did you see instead?

Editor and settings

"go.testFlags": [],
"go.coverOnSingleTest": true,
"go.coverOnSingleTestFile": true,
"go.coverOnTestPackage": true,
"go.useLanguageServer": true,
"go.useCodeSnippetsOnFunctionSuggest": true,
"go.formatTool": "gofumpt",
"go.toolsManagement.autoUpdate": true,
"go.autocompleteUnimportedPackages": false,
"[go]": {
  "editor.insertSpaces": false,
  "editor.autoClosingBrackets": "always",
  "editor.formatOnSave": true,
  "editor.codeActionsOnSave": {
    "source.organizeImports": true
  },
  "editor.suggest.snippetsPreventQuickSuggestions": false,
  "editor.semanticHighlighting.enabled": true
},
"gopls": {
  "formatting.gofumpt": true,
  "build.experimentalWorkspaceModule": true,
  "ui.noSemanticString": true,
  "ui.noSemanticNumber": true,
  "ui.semanticTokens": true,
  "ui.diagnostic.staticcheck": true,
  "ui.completion.matcher": "Fuzzy",
  "analyses": {
    "ST1000": false,
    "ST1003": false,
    "SA5001": false,
    "nilness": true,
    "unusedwrite": true,
    "fieldalignment": false,
    "shadow": false,
    "composites": false
  },
  "ui.codelenses": {
    "generate": false,
    "regenerate_cgo": true,
    "test": true,
    "vendor": true,
    "tidy": true,
    "upgrade_dependency": true,
    "gc_details": true
  },
  "ui.diagnostic.annotations": {
    "bounds": true,
    "inline": true,
    "escape": true,
    "nil": true,
  },
},

Logs

[findleyr]

CC @pjweinb

[pjweinb]

Thanks for reporting this. We need more information: what's the definition of 'routers'? Is it a type, a variable, or what?

[danicc097]

In the example all are variables. To be more specific, consider:

variables in the block should be tokenized as variable.parameter as well. It's behavior already present in Pylance, for example

[pjweinb]

1. According to the spec a semantic token could be either one of 'parameter' or 'variable', but not both.
2. Here is a file, like your example, with the semantic tokenization inserted as comments. (The comment has the length of the language token, the semantic token assigned, and a slice of semantic token modifiers.) What should be different?

/⇒4,keyword,[]/func /⇒1,function,[definition]/f[/⇒1,parameter,[definition]/T /⇒10,type,[]/comparable](/⇒1,parameter,[definition]/s []/⇒1,typeParameter,[]/T) /⇒5,type,[]/error {
/⇒3,variable,[definition]/set /⇒2,operator,[]/:= /⇒4,function,[defaultLibrary]/make(/⇒3,keyword,[]/map[/⇒1,typeParameter,[]/T]/⇒6,keyword,[]/struct{})
/⇒3,keyword,[]/for _, /⇒1,variable,[definition]/e := /⇒5,keyword,[]/range /⇒1,variable,[]/s {
/⇒2,keyword,[]/if _, /⇒2,variable,[definition]/ok /⇒2,operator,[]/:= /⇒3,variable,[]/set[/⇒1,typeParameter,[]/e]; /⇒2,variable,[]/ok {
/⇒6,keyword,[]/return /⇒3,namespace,[]/fmt./⇒6,function,[]/Errorf(/⇒8,string,[]/"bad %T", /⇒1,typeParameter,[]/e)
}
/⇒3,variable,[]/set[/⇒1,typeParameter,[]/e] /⇒1,operator,[]/= /⇒6,keyword,[]/struct{}{}
}
/⇒6,keyword,[]/return /⇒3,variable,[readonly defaultLibrary]/nil
}

[danicc097]

Sorry I was referring to textmate scope of variable.parameter.
The semantic token should remain "parameter" inside the function. As of now it's variable, so I believe it should be:

/⇒4,keyword,[]/func /⇒1,function,[definition]/f[/⇒1,parameter,[definition]/T /⇒10,type,[]/comparable](/⇒1,parameter,[definition]/s []/⇒1,typeParameter,[]/T) /⇒5,type,[]/error {
/⇒3,variable,[definition]/set /⇒2,operator,[]/:= /⇒4,function,[defaultLibrary]/make(/⇒3,keyword,[]/map[/⇒1,parameter,[]/T]/⇒6,keyword,[]/struct{})
/⇒3,keyword,[]/for _, /⇒1,variable,[definition]/e := /⇒5,keyword,[]/range /⇒1,parameter,[]/s {
/⇒2,keyword,[]/if _, /⇒2,variable,[definition]/ok /⇒2,operator,[]/:= /⇒3,variable,[]/set[/⇒1,typeParameter,[]/e]; /⇒2,variable,[]/ok {
/⇒6,keyword,[]/return /⇒3,namespace,[]/fmt./⇒6,function,[]/Errorf(/⇒8,string,[]/"bad %T", /⇒1,typeParameter,[]/e)
}
/⇒3,variable,[]/set[/⇒1,typeParameter,[]/e] /⇒1,operator,[]/= /⇒6,keyword,[]/struct{}{}
}
/⇒6,keyword,[]/return /⇒3,variable,[readonly defaultLibrary]/nil
}

[gopherbot]

Change https://go.dev/cl/445095 mentions this issue: tools/gopls: argument in function bodies marked as parameter by semantic tokens