Fuzzing doesn't work (no cgo)

[josselin-c]

Hello, I have a fuzzing job that doesn't run when it should. At first it was because of CGO but now I have CGO disabled and it still doesn't works. I'm not sure how to debug this.

Env

$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/joss/projects/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.9.2/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.9.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/3g/vj270kpd74374rcl0yv9c5dm0000gn/T/go-build339987089=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

Description

Fuzzing this code fails:

package fuzz

import (
	_ "github.com/ethereum/go-ethereum/core"
)

func Fuzz(data []byte) int {
	return 0
}

I'm fuzzing using go-ethereum/core version bfdc0fa3622d7c3b421d2f5a6dda5746be41bfde
I've stubbed the crypto/secp256k1 package as follow: https://gist.github.com/josselin-c/9c66bac044e1489c290c37c43250ff28

I'm building the fuzzing code with cgo disabled: `CGO_ENABLED=0 go-fuzz-build github.com/ethereum/go-ethereum/tests/vmfuzz``

The output:

go-fuzz -bin=./fuzz-fuzz.zip -workdir=results
2017/11/09 08:46:09 slaves: 4, corpus: 1 (3s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s
2017/11/09 08:46:12 slaves: 4, corpus: 1 (6s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 6s
2017/11/09 08:46:15 slaves: 4, corpus: 1 (9s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 9s
2017/11/09 08:46:18 slaves: 4, corpus: 1 (12s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 12s

[dvyukov]

I can't build this either way:

go-fuzz$ go-fuzz-build github.com/dvyukov/go-fuzz/examples/ethereum
failed to execute go build: exit status 2
# github.com/ethereum/go-ethereum/crypto/secp256k1
/tmp/go-fuzz-build699093233/gopath/src/github.com/ethereum/go-ethereum/crypto/secp256k1/curve.go:44:44: fatal error: libsecp256k1/include/secp256k1.h: No such file or directory
 #include "libsecp256k1/include/secp256k1.h"
                                            ^
compilation terminated.

go-fuzz$ apt-cache search secp256

go-fuzz$ CGO_ENABLED=0  go-fuzz-build github.com/dvyukov/go-fuzz/examples/ethereum
failed to execute go build: exit status 1
go build github.com/ethereum/go-ethereum/crypto/secp256k1: build constraints exclude all Go files in gopath/src/github.com/ethereum/go-ethereum/crypto/secp256k1

[dvyukov]

Try to send SIGABRT to few go-fuzz subprocesses. Then it should detect crashes and save stacks in workdir/crashes. Then send SIGABRT to main go-fuzz process.

[josselin-c]

For the build that doesn't work: you need to apply the patch from https://gist.github.com/josselin-c/9c66bac044e1489c290c37c43250ff28

I'll try to send some SIGABRT when I get back home 👍

[dvyukov]

There is something very weird happens. I could reproduce it once, but not anymore.
Somehow in the broken binary some other function was called instead of the desired Fuzz function (namely, Fuzz from mdlayher/ethernet/fuzz.go). That other function is not instrumented and that caused the hang.
I've renamed the package from fuzz to ethereum and it fixed the problem.
But then I renamed it back to fuzz and could not reproduce it any more.

As a stop-gap try to rename the package and add some real code to the Fuzz function.

FTR, I used go version devel +0ba4eba864 Mon Nov 6 08:37:30 2017 +0000 linux/amd64.

[dvyukov]

More debugging:

This seems to happen because "net" imports package "C". But "C" is not listed in Deps, while listed in Imports. As the result graph scheduling breaks -- net and everything that depends on it is not instrumented.

We could spoof "C" package in the work graph (setting FakeImportC in types.Config seem to fix it as well), but then typechecking of net breaks (it does not the real "C" package). To fix this, we would need to mimic more of go command logic and create/build real "C" package.

Which again reminds me of golang/go#19109 we can't chase go command infinitely.

[dvyukov]

Here is debugging patch that may be useful in future:
80caf48

I don't see how we can create and typecheck C package.
Of golang/go#19109 we need at least support for source-to-source transformation that works.

[josharian]

I’d be curious to know whether the recent upgrades to go-fuzz-build resolve this issue (either with cgo enabled or disabled).

[holiman]

It seems not (just ran into this and found this ticket)

[josharian]

@holiman can you provide new reproduction steps? I tried the OP's instructions and found that the patch did not apply cleanly.

[holiman]

I've tried adding a fuzzer to just a tiny part of go-ethereum, just add a fuzzer to some subdirectory and try to build it -- it will spit out the same error.
Using the CGO_ENABLED=0 gives me

[user@work core]$ CGO_ENABLED=0 go-fuzz-build
could not load packages: go [list -e -json -compiled=true -test=false -export=false -deps=true -find=false -tags gofuzz -- . github.com/dvyukov/go-fuzz/go-fuzz-dep]: exit status 1: go build github.com/ethereum/go-ethereum/crypto/secp256k1: build constraints exclude all Go files in /home/user/go/src/github.com/ethereum/go-ethereum/crypto/secp256k1

[josharian]

@holiman we know that go-fuzz doesn't work with cgo enabled. But that error message just means that github.com/ethereum/go-ethereum/crypto/secp256k1 cannot be compiled without cgo. The original poster stubbed out that package, so that it could be compiled without cgo, but the patch no longer applies cleanly. The ideal thing to do would be to have a non-cgo version available in the ethereum tree, if there is a pure Go implementation of it. Failing that, it might be worth having stubs in the ethereum tree, but that is up to them not me. In any case, until there is a non-cgo version that compiles, go-fuzz-build won't work.

[holiman]

@josharian but there is a non-cgo version of it. The problem was that a couple of culprits in the codebase referenced directly into the cgo-version, instead of referencing into the common one. See the PR that is referenced above.: ethereum/go-ethereum#19593

I'm running several fuzzers now.

[josharian]

Ah! So does that mean that this issue can now be closed?

[holiman]

Yes, I think this issue can be closed (at least once we remove the remaining references that poke into C-land, spread a bit here and there (scwallet, ipc_unix etc)).