Sleep should not use one thread per running call #1644
Labels
Comments
Oops, sorry, the correct patch is here. Attachments:
|
GOMAXTHREADS will not happen. We need fewer environment variables, not more. This is basically working as intended: if you block 132 threads in system calls then Go will create a 133'rd so that it can keep doing useful work. If there were a limit the program would just hang mysteriously, which is just as bad as crashing, if not worse. The real problem is that time.Sleep should not create a new thread for each sleep. This has been discussed on the list before. Owner changed to @rsc. Status changed to Accepted. |
I totally agree now, the real problem was Go allocating ~1TB of virtual memory per OS thread (although I think it was actually exponential, since stack address was going higher and higher) which meant it was running out of memory and couldn't create additional threads. See https://groups.google.com/d/msg/golang-nuts/tSDpLubyVDo/7fBbYVE4IxsJ for details. The real hard-limit on number of threads on OS X is 2560, which is big enough for all practical reasons. Of course non-blocking time.Sleep would be awesome. :) |
|
Comment 4 by webmaster@webmaster.ms: It is almost done on Windows. The idiomatic solution of asynchronous timers (used, for example, in boost::asio) is to specify the next timer's remaining time as the GetQueuedCompletionStatus() timeout. Return from GetQueuedCompletionStatus() wakes up a goroutine which had been waited either for an IO completion or for a timer event. There is already an IOCP thread in src\pkg\net\fd_windows.go with infinite timeout, and it seems trivial to reuse it for the timing purposes. As far as I can remember, epoll can be used the similar way. |
|
This issue was closed by revision 3b86026. Status changed to Fixed. |
This issue was closed.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
When there are many goroutines that are likely to block on syscalls new OS threads are spawned uncontrollably, which can lead to whole program going down if new thread cannot be created anymore. For example, on Mac OS X 10.6 go fails to create 132nd thread. There needs to be a way to limit maximum number of OS threads that go can create to doesn't get out of control. The attached patch introduces GOMAXTHREADS environment variable that limits maximum number of threads. The minimum OS threads that simple go programs need appears to be 2 (that's why this minimum is hardcoded), to successfully complete the test suite GOMAXTHREADS needs to be at least 3. If not specified or 0 (default) then the behavior doesn't change and means "unlimited". The patch is very simple and not very efficient. For example it drains sched queue and if there are many scheduled goroutines, but small number of OS threads, constant rescheduling is likely a performance problem, but I just didn't want to change existing functions too much, and it's more proof of concept. There's also a problem when runtime·iscgo is true it leads to runtime·throw("all goroutines are asleep - deadlock!"), I don't know why, so when runtime·iscgo is true my patch simply uses unlimited number of OS threads. A program that can illustrate the problem is very simple: package main import ( "fmt" "time" ) func main() { for i := 0; i < 256; i++ { go func(i int) { fmt.Printf("Sleeping in %d\n", i) time.Sleep(60*1e9) }(i) <- time.After(100*1e6) } }Attachments:
The text was updated successfully, but these errors were encountered: