Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: track time or counts goroutine entered runtime #6809

Closed
bradfitz opened this issue Nov 21, 2013 · 10 comments
Closed

runtime: track time or counts goroutine entered runtime #6809

bradfitz opened this issue Nov 21, 2013 · 10 comments
Labels
FrozenDueToAge

Comments

@bradfitz
Copy link
Contributor 

For debugging goroutine leaks, it would be nice if the runtime could tell me how long
each goroutine had been stuck in its current state.

e.g. a goroutine blocked on a lock or channel send/recv for hours is probably busted.

Could that timestamp be kept per-G?  It's not much more memory.

The time need not be perfectly accurate, either (if getting time every entry into
runtime is viewed as too expensive), since the point I at least care about is just
seeing very old things.

Alternatively, all of this could be done in my own program (polling every ~30 seconds)
if I had a per-G unique value that changed whenever the goroutine state changed.  A
per-G counter would be enough, incremented every entry to the runtime.
@rsc
Copy link
Contributor

rsc commented Nov 21, 2013

Comment 1:

the sysmon goroutine that deals with things like preemptions could probably
record this information for stopped goroutines, and then the goroutine
could just set it back to 0 when it started again.

@dvyukov
Copy link
Member

dvyukov commented Nov 22, 2013

Comment 2:

A better way to do this is during GC. GC walks all goroutines, looks at their statuses
and queries time anyway. So this will be a few cheap lines in GC. GC is forced every 5
min, so this info will be populated even in a completely stalled process.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 3:

Labels changed: added release-none, removed go1.3maybe.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 4:

Labels changed: added repo-main.

@dvyukov
Copy link
Member

dvyukov commented Jan 9, 2014

Comment 5:

It's quite easy to implement and it must have negligible overheads, so the main question
is whether we want it at all or not.
Sameer, Sougou, you are serious users of Go in the wild, would it be useful to output in
all stack traces something like the following?
goroutine 739 [IO wait, blocked for 2 minutes]:
net.runtime_pollWait(0x28837d28, 0x72, 0x0)
        /tmp/gobuilder/freebsd-386-e373697cd435/go/src/pkg/runtime/netpoll.goc:116 +0x5e
net.(*pollDesc).Wait(0x399eb078, 0x72, 0x28836188, 0x23)
        /tmp/gobuilder/freebsd-386-e373697cd435/go/src/pkg/net/fd_poll_runtime.go:81 +0x32
goroutine 743 [chan receive, blocked for 2345 minutes]:
net/http.(*persistConn).readLoop(0x39b0dd70)
        /tmp/gobuilder/freebsd-386-e373697cd435/go/src/pkg/net/http/transport.go:778 +0x543
created by net/http.(*Transport).dialConn
        /tmp/gobuilder/freebsd-386-e373697cd435/go/src/pkg/net/http/transport.go:528 +0x53b

@minux
Copy link
Member

minux commented Jan 9, 2014

Comment 6:

IMO, that will be really useful.

@bradfitz
Copy link
Contributor Author

bradfitz commented Jan 9, 2014

Comment 7:

I filed the bug, but I will reiterate that this would be very useful.

@sougou
Copy link
Contributor

sougou commented Jan 9, 2014

Comment 8:

I can think of one instance in the past where this info would have been handy. I even
sent dvyukov a stack trace about it, when vttablet hung due to SSL thundering herd.
In that situation, we got to the bottom of it through indirect means.

@dvyukov
Copy link
Member

dvyukov commented Jan 10, 2014

Comment 9:

Mailed
https://golang.org/cl/50420043

Owner changed to @dvyukov.

Status changed to Started.

@dvyukov
Copy link
Member

dvyukov commented Jan 16, 2014

Comment 10:

This issue was closed by revision c0b9e62.

Status changed to Fixed.

@golang golang locked and limited conversation to collaborators Jun 25, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@bradfitz @rsc @minux @sougou @dvyukov @gopherbot