cmd/go: trim go.sum without adding module requirements #29800
Comments
bradfitz
added
FeatureRequest
bcmills
changed the title
bcmills
changed the title
bcmills
added
the
NeedsDecision
|
It would be nice to have a way to cleanup go.sum from stuff that are not relevant for the building the module and only keep the minimal required. The current behavior adds hashes for all the stuff in one's go root which one might not want to expose to others (privacy) when pushing the module to a repo. Right now I need to manually cleanup the go.sum file before pushing it to the repo. relevant bug: #26381 |
|
Any update on this? Am I right in understanding that there is still no way to trim hashes from |
|
@qbiqing This issue is about having separate functionality to trim sums from Since this is a very narrow case, and since having additional hashes in |
|
|
I wrote some code to do this. |
|
here is the hacky way I did it:
// Binary gosumtrim trims the go.sum file based on the given go.mod file.
package main
import (
"bufio"
"flag"
"fmt"
"io"
"os"
"strings"
"github.com/golang/glog"
)
var (
modFile = flag.String("mod", "", "input go.mod file")
sumFile = flag.String("sum", "", "input go.sum file")
outFile = flag.String("out", "", "output go.sum file")
)
func main() {
flag.Parse()
modIn, err := os.Open(*modFile)
if err != nil {
glog.Exitf("Open(%q) failed: %v", *modFile, err)
}
sumIn, err := os.Open(*sumFile)
if err != nil {
glog.Exitf("Open(%q) failed: %v", *sumFile, err)
}
sumOut, err := os.Create(*outFile)
if err != nil {
glog.Exitf("Create(%q) failed: %v", *outFile, err)
}
if err := trim(modIn, sumIn, sumOut); err != nil {
glog.Exitf("Trim() failed: %v", err)
}
if err := sumOut.Close(); err != nil {
glog.Exitf("os.Close() fialed: %v", err)
}
}
func trim(mod io.Reader, sum io.Reader, o io.Writer) error {
m := bufio.NewReader(mod)
s := bufio.NewReader(sum)
// Read till we get to the require line.
for {
line, err := m.ReadString('\n')
if err != nil {
return fmt.Errorf("reading mod file failed: %v", err)
}
glog.Infof("line: %v", line)
if line == "require (\n" {
break
}
}
// Process require lines and add dependencies into used map.
used := make(map[string]bool)
for {
line, err := m.ReadString('\n')
line = strings.Trim(line, "\n \t")
if err != nil {
return fmt.Errorf("reading input go.mod failed: %v", err)
}
glog.Infof("line: %v", line)
if line == ")" {
break
}
used[line] = true
}
glog.Infof("used: %v", used)
// Filter sum->out, dropping what is not used in mod.
for {
line, err := s.ReadString('\n')
if err == io.EOF {
break
}
glog.Infof("line: %v", line)
if err != nil {
return fmt.Errorf("reading input go.sum failed: %v", err)
}
parts := strings.Split(strings.TrimSpace(line), " ")
if len(parts) != 3 {
return fmt.Errorf("expecting 3 parts per go.sum line, got %v", parts)
}
if !used[strings.Join(parts[:2], " ")] {
continue
}
if _, err := o.Write([]byte(line)); err != nil {
return fmt.Errorf("writing output go.sum failed: %v", err)
}
}
return nil
}
package main
import (
"bytes"
"io"
"io/ioutil"
"os"
"testing"
"github.com/google/go-cmp/cmp"
)
const (
root = ".../gosumtrim/" // module root
inModFile = root + "testdata/go.mod.in"
inSumFile = root + "testdata/go.sum.in"
outSumFile = root + "testdata/go.sum.out"
)
func TestTrim(t *testing.T) {
mod := mustOpen(t, inModFile)
sum := mustOpen(t, inSumFile)
out := mustOpen(t, outSumFile)
want, err := ioutil.ReadAll(out)
if err != nil {
t.Fatalf("ioutil.ReadAll(out) failed: %v", err)
}
b := bytes.NewBuffer(nil)
if err := trim(mod, sum, b); err != nil {
t.Fatalf("trime() failed: %v", err)
}
got := b.Bytes()
t.Logf("got:\n%v", string(got))
t.Logf("want:\n%v", string(want))
if diff := cmp.Diff(string(got), string(want)); diff != "" {
t.Fatalf("trime() returned:\n%s", diff)
}
}
func mustOpen(t *testing.T, fn string) io.Reader {
t.Helper()
r, err := os.Open(fn)
if err != nil {
t.Fatalf("os.Open(%q) failed: %v", fn, err)
}
return r
} |
|
|
What version of Go are you using (
go version)?go version go1.11.4 linux/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env)?What did you do?
go mod init github.com/F21/somerepo.go.modandgo.sumby runninggo test ./....github.1485827954.workers.dev/uber/jaeger-client-goconflicts with another dependency, so I bumped it to master:go get github.com/uber/jaeger-client-go@master.github.com/uber/jaeger-client-goin my go.sum:The
github.com/uber/jaeger-client-go v2.15.0+incompatibleentries are unnecessary as the project is now pinned github.com/uber/jaeger-client-go v2.15.1-0.20190116124224-6733ee486c78+incompatible (current master).In #26381, the ability to trim
go.sumwas added togo mod tidy. However,go mod tidyis something I avoid as it pulls in all test dependencies for all my dependencies and their dependencies for all platforms. Unfortunately,go mod tidyhas burnt me quite a few times and causes a lot more problems than it solves.See:
What did you expect to see?
I'd like to be able to trim unneeded entries from go.sum without having to run
go mod tidyas its current behavior causes a lot more problems than it solves.What did you see instead?
N/A.
The text was updated successfully, but these errors were encountered: