-
Notifications
You must be signed in to change notification settings - Fork 18k
runtime: fatal error: sweep increased allocation count in go1.7 #16778
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Yeah, "don't do that then. you're in wacky land because of unsafe" The rules around unsafe, cgo, etc to be nice to the increasingly-capable GC are getting stricter with each release. If you run it with GOGC=1 it'll crash even sooner. |
Ok so for clarity, you are no longer allowed to create/inspect pointer values that aren't pointing at valid Go allocated memory anymore? I'm not referring to dereferencing them, that was always in wacko land, just handling them. |
Seems like the documentation at https://golang.org/pkg/unsafe/ specifically says this is invalid under bullet point (3), so I'll consider this fully answered. Thanks. |
Whether they point to Go allocated memory or not is irrelevant. You have always been able (and continue to be able) to have Go pointers point to differently allocated memory (globals, cgo-allocated stuff, syscall.Mmap, etc.). What's currently newly breaking in go 1.7 is pointer arithmetic on pointers that point to no memory, i.e. faulting pages. |
#16772 #16772 is about pointers Expect the GC to continue to get better at spotting and reporting these On Wed, Aug 17, 2016 at 11:56 PM, Keith Randall [email protected]
|
I'm having this issue with all go packages on arch linux, I notice because every time i google the error i land on this issue, with go package version: 2:1.7.3-1 I don't have any kind of MAC or RBAC such as SELinux enabled. Could someone perhaps post a example "bad code" and how to fix it, for non-golang-specialists ? That would be super nice 🎸 Edit: I ran the same program again and it didn't crash. |
I'm having this issue after upgrade to go 1.7 |
It would be great if you could create a reproducer. Next confirm it doesn't Without a reproducer or some more clues this is not really actionable. On Sat, Oct 29, 2016 at 11:27 AM, Icecream [email protected] wrote:
|
I seem to have a similar issue with 1.7.3 linux/amd64. In my case, I was porting a perl script to go and ended up with something like https://gist.github.com/qrwteyrutiyoup/97adfd0a26f155317df7354351a35475 Problem #1 was during "go build". Sometimes it would work OK, some other times it would throw errors like https://gist.github.com/qrwteyrutiyoup/1a74373eb24667f06031df79151c86d4 Problem #2 was running the generated program itself. Sometimes it would work OK, and some other times, throw errors like these: https://gist.github.com/qrwteyrutiyoup/5c1746150aa5fa26b83f8b1ca9d44e67 (similiar to the previous one) To test the program, you can use this test file https://www.uece.net/misc/test-file.txt.xz (compressed due to size; ~32MB uncompressed; about 1.1M compressed), passing it as argument to the program. Expected output: Let me know if I can provide additional info to debug the issue (or if I am doing something silly in the code itself) |
Thanks for the replicator. Unfortunately, the replicator reliable produces the expected output on my laptop. I have not been able to reproduce the reported error. I will leave it running overnight. Perhaps your machine has many more HW threads than mine. Could you try and reproduce it with GOMAXPROCS=4? |
Thanks for checking it out. It did reproduce here, although not as reliably, indeed. Try to put it in an infinite loop and it crashes more quickly: I tried this in loop (both for go build and the program execution itself, and it started crashing in go 1.5.x (which is when the compiler started to be written in go?). Go 1.4 doesn't crash at all, loop or no loop. Tested with the binary distributions available for download on the Go website. |
Interesting. I am using the 1.7.3 binary from golang.org and I've been running it in the loop as you suggested. I've also tried GOGC=1 without a failure. I'll keep at it. |
Oh, I see. I was using 1.7.3 from ArchLinux; the older versions I was using from golang.org, but I just checked with the 1.7.3 binary from there and got the same result. Starting to look like something specific to this computer, but it's odd that it works fine with Go 1.4.3. My current version of the sample program I posted still crashes in 1.7.3 (and not in 1.4.3), but interestingly, it gave me already 3 different crashes:
|
OK, I did some more testing and probably found the culprit. Here's the findings:
Interestingly, it crashed in my 2 ArchLinux boxes, a desktop and a laptop. After checking out what could be different in such systems, I realized I was using a -ck kernel (https://wiki.archlinux.org/index.php/linux-ck). I returned to a "stock" kernel (4.8.6-1-ARCH) and was unable to reproduce the issue since then.
So, I am not sure this is an issue that should be documented/investigated, or if it's a problem with the kernel itself, but for now I am good, on a different kernel. |
And I may have pulled the trigger too quickly, as I just noted the other computer which reproduced the problem had vanilla kernel 4.8.0-gc8d2bc9 on it. I can't change its kernel right now, but tomorrow I will give it a try with the same version I have on the laptop now, to see if it the crash goes away. |
@qrwteyrutiyoup thanks! I have two ArchLinux boxes with -ck kernel and your comment really brings my a hope=) go compiler and syncthing is broken. I will try to use stock |
Still reproducible on 4.8.7, but works on vanilla 4.8.10 on Arch Linux thanks a heap !! However, 4.8.10 doesn't have CONFIG_USER_NS, building a 4.8.10 with it and will report back if it reproduces Kudos to @qrwteyrutiyoup for the epic investigation logs 🎸 |
@jpic did you try with
|
I have it enabled right now on a 4.9.6-1-userns kernel and I'm experiencing
I don't think that's really a golang bug though, looks like an acbuild/cobra bug. |
Please answer these questions before submitting your issue. Thanks!
go version
)?go version go1.7 darwin/amd64
go env
)?GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH=""
GORACE=""
GOROOT="/Users/jeff/go"
GOTOOLDIR="/Users/jeff/go/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/tn/09dglcts0b9gc5111hnf0_dm0000gn/T/go-build421610542=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
I ran this program: https://play.golang.org/p/LXVU7fsuOr
A bunch of pointers forever.
A bunch of pointers and then
I'm aware this program is doing something silly by constructing a pointer to a field that isn't backed by any allocation, but it didn't crash on go1.6 and I'm not sure if there's any defined behavior around this. If the answer is "don't do that then. you're in wacky land because of unsafe", that's fine, I just want to be sure that this isn't a problem. I found it interesting that it took many prints for it to fail, rather than failing right away.
The text was updated successfully, but these errors were encountered: