Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spawn many isolates: better errors if too many, scale up #2598

Closed
sigmundch opened this issue Apr 16, 2012 · 14 comments
Closed

spawn many isolates: better errors if too many, scale up #2598

sigmundch opened this issue Apr 16, 2012 · 14 comments
Labels
area-vm Use area-vm for VM related issues, including code coverage, FFI, and the AOT and JIT backends. closed-obsolete Closed as the reported issue is no longer relevant type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)

Comments

@sigmundch
Copy link
Member

I'm creating this bug to track progress on this issue:
 - we need to report better error messages when we can't create another isolate (e.g. an exception like 'TooManyIsolates')
 - we eventually want to scale to a larger number of isolates.


(original bug as reported in misc@)

I can only spawn 79 isolate, is there any bug in below code or any
limitation on dart vm ?

What steps will reproduce the problem?

run below script on Linux with latest dart SDK.

import('dart:isolate');

myfun() {
 port.receive( (msg, SendPort reply) {
   print("recieve: $msg");
   reply.send("received");
 });
}

void test() {

 SendPort port = spawnFunction( myfun );
 port.call("hello").then( (reply) {
   print(reply);
 });

}

main()
{

 var times = 80;

 Stopwatch w = new Stopwatch();
 w.start();

 for(int i=0; i<times; i++) {
   test();
 }
 w.stop();

 print("time used: ${w.elapsedInMs()}");

}

./runtime/vm/scavenger.cc:162: error: expected: space_ != NULL

155 Scavenger::Scavenger(Heap* heap, intptr_t max_capacity, uword
object_alignment)
156 : heap_(heap),
157 object_alignment_(object_alignment),
158 count_(0),
159 scavenging_(false) {
160 // Allocate the virtual memory for this scavenge heap.
161 space_ = VirtualMemory::Reserve(max_capacity);
162 ASSERT(space_ != NULL);

ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 20
file size (blocks, -f) unlimited
pending signals (-i) 16382
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

@kasperl
Copy link

kasperl commented Apr 17, 2012

I'm marking this as a VM issue for now.


Added Area-VM, Triaged labels.

@sigmundch
Copy link
Member Author

Added Isolates label.

@iposva-google
Copy link
Contributor

Set owner to @mraleph.

@sigmundch
Copy link
Member Author

Added Library-Isolates label.

@sigmundch
Copy link
Member Author

Removed Isolates label.

@mraleph
Copy link
Member

mraleph commented Sep 7, 2012

It is still reproducible, though now I can have ~130 isolates.

I am not sure if getting an error instead of a crash should be scheduled for M1.

What do you think Todd?


Set owner to @turnidge.

@dgrove
Copy link
Contributor

dgrove commented Jan 11, 2013

Added Library-Isolate label.

@dgrove
Copy link
Contributor

dgrove commented Jan 11, 2013

Removed Library-Isolates label.

@iposva-google
Copy link
Contributor

Removed the owner.
Added this to the Later milestone.
Removed Library-Isolate label.

@iposva-google
Copy link
Contributor

Removed Priority-Medium label.
Added Priority-Unassigned label.

@iposva-google
Copy link
Contributor

Waiting for the new isolate library, which should allow error reporting when spawning isolates.

@DartBot
Copy link

DartBot commented Feb 11, 2014

This comment was originally written by Erik.G...@gmail.com


I found this benchmark of isolate performance in the misc group https://groups.google.com/a/dartlang.org/forum/#!msg/vm-dev/WxhM23rYhig/WNiGHYkBudcJ.

I performed a basic benchmark to explore the performance of an isolate per request model. In dart it takes between 22 and 30 ms from the time Isolate.spawn is called to the time the entryPoint method is actually called (measured by sending new DateTime.now to the child isolate). The equivalent java version takes 1 ms or less. The basic gist is that the creating isolates is likely too costly for a per request model (where the servers portion of response budget is < 100 ms). I did a basic warm up for both vm's spinning up 1000 isolates/threads.

@kasperl
Copy link

kasperl commented Jul 10, 2014

Removed this from the Later milestone.
Added Oldschool-Milestone-Later label.

@kasperl
Copy link

kasperl commented Aug 4, 2014

Removed Oldschool-Milestone-Later label.

@sigmundch sigmundch added Type-Defect area-vm Use area-vm for VM related issues, including code coverage, FFI, and the AOT and JIT backends. labels Aug 4, 2014
@kevmoo kevmoo added type-bug Incorrect behavior (everything from a crash to more subtle misbehavior) and removed Type-Defect labels Mar 1, 2016
@iposva-google iposva-google added the closed-obsolete Closed as the reported issue is no longer relevant label May 19, 2016
dart-bot pushed a commit that referenced this issue Aug 11, 2020
> git log --oneline 04b054b62cc437cf23451785fdc50e49cd9de139..master
0d185a39 (HEAD -> master, origin/master, origin/HEAD) Push null-safety forwards to 2.11 (#2604)
56f9f27f Hide outdated --mode flag (#2603)
61ce6f81 Avoid double loop (#2605)
fa6e57d7 (disable_mixed_mode_validation) Fix outdated latest ordering (#2598)
6549e4aa Remove unused dependency from pubspec.yaml (#2592)
61543d07 Don't look for external package foo during testing (#2599)
590b448f Fixed license headers (#2595)
04e0601e Don't show entries for dev-dependencies in outdated --json --no-dev-dependencies (#2591)
8c3778c4 Configure GitHub move app (#2578)
eec7beca (top_level_command) Pass --(no-)sound-null-safety arg through to VM. (#2542)
152e4740 Warn about publishing in mixed mode (#2583)
0b7a3abe Removed april fools toys (#2325)
b74a5b73 Actually print hints (#2582)
8ec3a66d (pub2) Fix outdated --no-color (#2572)
7bb3d4e6 Use getSdkPath() in NullSafetyAnalysis (#2573)
3c578f24 Drop the "magic" package concept (#2577)
0e967ff0 Remove unused function (#2570)
988fefef Remove dependency overrides (#2568)

Change-Id: I58bf14234ed55bf9d825de60a40ded1d65281195
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158003
Reviewed-by: Jonas Jensen <jonasfj@google.com>
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-vm Use area-vm for VM related issues, including code coverage, FFI, and the AOT and JIT backends. closed-obsolete Closed as the reported issue is no longer relevant type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)
Projects
None yet
Development

No branches or pull requests

7 participants