Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Out of memory in test.dart #14761

Closed
peter-ahe-google opened this issue Nov 4, 2013 · 8 comments
Closed

Out of memory in test.dart #14761

peter-ahe-google opened this issue Nov 4, 2013 · 8 comments
Labels
area-infrastructure Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes. P2 A bug or feature request we're likely to work on type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)

Comments

@peter-ahe-google
Copy link
Contributor

Check out r29345.

Patch in https://codereview.chromium.org/57773002 (this CL prints a lot of debug information to stdout).

Run:

./tools/test.py -mrelease -cdart2js -rd8 --time -pcolor --report --failure-summary --checked

This leads to this crash in test.dart:

Test configuration: dart2js_d8_release_ia32_checked
[00:47 | --% | + 419 | - 0]Total: 10058 tests
 * 107 tests will be skipped (7 skipped by design)
 * 10 tests are expected to be flaky but not crash
 * 8991 tests are expected to pass
 * 28 tests are expected to fail that we won't fix
 * 552 tests are expected to fail that we should fix
 * 2 tests are expected to crash that we should fix
 * 9 tests are allowed to timeout
 * 0 tests are skipped on browsers due to compile-time error

[11:28 | 45% | + 4536 | - 0]Exhausted heap space, trying to allocate 268435472 bytes.
Uncaught Error: Out of Memory
Stack Trace:

­0 List._grow (dart:core-patch/growable_array.dart:166:24)

­1 List.add (dart:core-patch/growable_array.dart:129:12)

­2 addAll (dart:core-patch/growable_array.dart:136:10)

­3 _BufferingStreamSubscription._sendData.<anonymous closure> (dart:async/stream_impl.dart:184:55)

­4 _ZoneBase._runInZone (dart:async/zone.dart:82:17)

­10 _ZoneBase._runGuarded (dart:async/zone.dart:99:22)

­11 _ZoneBase.executePeriodicCallbackGuarded (dart:async/zone.dart:68:21)

­12 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:184:41)

­13 _BufferingStreamSubscription._add (dart:async/stream_impl.dart:135:16)

­14 _StreamController&_SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:198:23)

­15 _StreamController._add (dart:async/stream_controller.dart:130:16)

­16 _StreamController.add (dart:async/stream_controller.dart:104:9)

­17 _onData (dart:io-patch/socket_patch.dart:827:42)

­18 _BufferingStreamSubscription._sendData.<anonymous closure> (dart:async/stream_impl.dart:184:55)

­19 _ZoneBase._runInZone (dart:async/zone.dart:82:17)

­20 _ZoneBase._runGuarded (dart:async/zone.dart:99:22)

­21 _ZoneBase.executePeriodicCallbackGuarded (dart:async/zone.dart:68:21)

­22 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:184:41)

­23 _BufferingStreamSubscription._add (dart:async/stream_impl.dart:135:16)

­24 _StreamController&_SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:198:23)

­25 _StreamController._add (dart:async/stream_controller.dart:130:16)

­26 _StreamController.add (dart:async/stream_controller.dart:104:9)

­27 _RawSocket._RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:544:52)

­28 _NativeSocket.multiplex (dart:io-patch/socket_patch.dart:344:39)

­29 _NativeSocket.connectToEventHandler.<anonymous closure> (dart:io-patch/socket_patch.dart:429:54)

­30 _ReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:81:92)

Unhandled exception:
Out of memory
@mkustermann
Copy link
Member

The exception tells us that it is allocating 256MB just for one list. It is most likely accumulating bytes in that list for dart2js output. We're driving test.dart with a 32-bit executable, which would mean that the output of dart2js is between 32MB-64MB in that case.

Having one test output 32MB+ is not anticipated in test.dart.

IMHO: For test.dart it doesn't make much sense to collect output of that size (since test.dart's failure summary will be unreadable [except if it is piped into a file, but still]).

We had a similar issue some time ago where a test was printing an infinite amount of output.

So I propose introducing a limit on the number of bytes we collect from a running test. After hitting that limit we discard the remaining data.
How does that sound?

@sgjesse
Copy link
Contributor

sgjesse commented Nov 4, 2013

I think setting a limit and truncating the collected stdout/stderr is the right approach.

@peter-ahe-google
Copy link
Contributor Author

Would it be possible to keeping the last n bytes of output?
Often it is the last part that is important as it contains a stack trace.

@mkustermann
Copy link
Member

This doesn't solve the problem when a test is misbehaving and printing an infinite of data. We could of course still take the first MB and the last MB and wait until it will be killed by test.dart due to a timeout.

cc @ricowind.
Removed Priority-Unassigned label.
Added Priority-Medium label.

@peter-ahe-google
Copy link
Contributor Author

Martin, what do you mean by "This doesn't solve the problem"?

A cyclic buffer of size n would record the last n bytes of output.

@ricowind
Copy link
Contributor

Removed Area-Test label.
Added Area-infrastructure label.

@lrhn
Copy link
Member

lrhn commented Jan 28, 2014

Removed Area-infrastructure label.
Added Area-Infrastructure label.

@peter-ahe-google peter-ahe-google added Type-Defect area-infrastructure Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes. labels Jan 28, 2014
@kevmoo kevmoo added P2 A bug or feature request we're likely to work on type-bug Incorrect behavior (everything from a crash to more subtle misbehavior) and removed triaged labels Feb 29, 2016
@bkonyi
Copy link
Contributor

bkonyi commented Jun 28, 2018

Has this been resolved? Closing for now, assuming it isn't relevant anymore.

@bkonyi bkonyi closed this as completed Jun 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-infrastructure Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes. P2 A bug or feature request we're likely to work on type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@kevmoo @sgjesse @lrhn @ricowind @peter-ahe-google @mkustermann @bkonyi