proposal: runtime: allow programs to handle deadlock events

[cwmos]

Summary

It would be great if a Go program could detect when all (other) Goroutines are blocked and then allow it to take action to do something about the situation. Given that Go has a deadlock detector, it seems like this situation is already detected by the Go runtime.

Use case

I want to use this to implement fake time that:

• Advances as fast as it can given the actual CPU resources available
• To the program that runs, makes it look like infinite CPU resources are available.

Implementing such fake time can basically be done as follows:

Loop:

• Wait until all (other) Goroutines are blocked
• Advance fake time to a point where a new event happens. For example, to a point where a channel returned by a fake version of time.After gets a new value written to it or a fake version of time.Sleep returns.

This is something I have previously successfully implemented in other programming languages and that has been very useful. There are several use cases of this including:

• Test timeouts quickly and accurately: Some programs have various (long) timeouts. To unit test those, it is an advantage to be able to advance time quickly and accurately.
• Test distributed systems: You may want to run a big distributed system written in Go in a single Go process running on a single computer using a fake network. There may not be enough CPU to do this in real time. Fake time as described here will allow this to run using all the CPU available and time will just progress as fast as it can. Fake time may then at any instant be either slower or faster than real time. I believe that this network simulator does something similar: https://www.nsnam.org/docs/dce/manual/html/getting-started.html
• Debug code that uses timers: Allow breakpoints to be set while debugging code that uses timers. Time and timers will automatically stop while the program is stopped in a breakpoint and resume when the program is started again.
• Detect unintended busy loops that only lasts for a limited amount of time: If fake time is used, such busy loop will run forever which will be easily detectable. I have found several bugs like this during the years by using fake time.

Some ways to make the feature

• Make a version of runtime.gosched that only returns when all other Goroutines are blocked.
• Make it possible to read the number of Goroutines that are not blocked. You could then make a loop where you call runtime.gosched until that value is 0.
• Make it possible to start a special Goroutine when the system is deadlocked.

A nuance

What to do if no Goroutine can run now, but one or more Goroutines are pseudo-blocked, for example:

• Waiting on external events, for example a socket
• Stuck in a cgo call
• Waiting on one or more channels returned by the real non-fake time.After function.

UPDATED: In a simple approach where the existing deadlock detector is used, any pseudo-blocked Goroutine would be considered non-blocked, and hence prevent fake time from increasing. The feature would still be very useful. However, a probably even more useful approach would be to consider pseudo-blocked Goroutines blocked. In that case, fake time would increase even in case of pseudo-blocked Go routines. If a program wants to prevent fake time from increasing because of a pseudo-blocked Goroutine it knows of, it can just prevent that manually (some details would need to be considered to make sure this can work correctly).

An off topic note

Should fake time not be an OS feature? Or a feature of a virtual machine hypervisor? Yes, I think absolutely it should!

[seankhliao]

see also #8869

[ianlancetaylor]

The majority of real Go programs read from the network or from pipes. Many also use timers. Some use the os/signal package to look for signals. Any Go program that does any of those things can never deadlock.

The Go runtime already supports fake time, although it is not fully supported. See https://golang.org/src/runtime/time_fake.go. That may be a more profitable approach than focusing on detecting deadlocks.

[beoran]

I don't think the fake time idea is very interesting, but having a way to try to resolve deadlock does seems to be useful, at least if we had a way to terminate the blocked goroutines, which IIRC, we don't have. Otherwise something like a runtime.SetDeadlockCallback(callback func(goroutineID ...int)) could be useful.

[cwmos]

The majority of real Go programs read from the network or from pipes. Many also use timers. Some use the os/signal package to look for signals. Any Go program that does any of those things can never deadlock.

I think you have an important point about networking. I think the feature will still be useful in case networking is used: First, networking may be mocked in code during testing. Second, I have added some more text in the proposal under "UPDATED" with ideas on how to address this.

Regarding timers, a central idea is that the Go program should use fake versions of timers. Hence real native Go timers would not be used and hence not prevent deadlock detection.

[cwmos]

Since I opened this issue I have used Go some more. I still think the requested feature would be extremely useful to mock time in tests as I have described previously. However, I want to mention a simpler application.

Suppose I have written this program:

type Result interface {
	Give(int)
}

func multiply(a, b int, result Result) {
	go func() {
		result.Give(a * b)
	}()
}

Suppose I want to test the multiply function using gomock. After having generated mocks for the Result interface, I could try to do it like this:

func TestMultiply(t *testing.T) {
	ctrl := gomock.NewController(t)
	defer ctrl.Finish()
	m := mock.NewMockResult(ctrl)
	m.EXPECT().Give(4)
	multiply(2, 2, m)
}

Unfortunately there is a race and the test may fail or succeed depending on whether the Give function is called before the test ends or not.

If we have the feature of this issue I could just add the line:

runtime.WaitForDeadlock()

at the end of the TestMultiply function and there will be no races.

Now, of course the problem can be solved by using channels or WaitGroups in the test (see for example https://medium.com/@poy/gomock-and-go-routines-6a7c01d989d5). However, that is cumbersome. Further, if multiply makes additional calls to Give after the test has finished, the test will still succeed even though it should not.

[rsc]

This seems like a fairly invasive, subtle runtime change that would rarely be useful in practice (because, as others have noted, programs with network listeners or readers never deadlock). It does not seem worth the weight.

[cwmos]

@rsc , yes this is really for unit tests and simulations where all network activity is mocked or faked so that no real sockets are used.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

Based on the discussion above, this proposal seems like a likely decline.
— rsc for the proposal review group

[rsc]

No change in consensus, so declined.
— rsc for the proposal review group