Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Polymer tests are flaky in Dartium #19196

Closed
vsmenon opened this issue Jun 4, 2014 · 19 comments
Closed

Polymer tests are flaky in Dartium #19196

vsmenon opened this issue Jun 4, 2014 · 19 comments
Labels
P2 A bug or feature request we're likely to work on type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)

Comments

@vsmenon
Copy link
Member

vsmenon commented Jun 4, 2014

We're not getting much useful tracking on these since the 35 roll. Not seeing anything similar outside these polymer tests.

John / Siggi: any ideas what might be different here? Have you seen anything similar in dart2js?

polymer/example/component/news/test/news_index_test: Pass, RuntimeError # Issue #8931
polymer/test/attr_deserialize_test: Pass, RuntimeError # Issue #18931
polymer/test/attr_mustache_test: Pass, RuntimeError # Issue #18931
polymer/test/bind_test: Pass, RuntimeError # Issue #18931
polymer/test/custom_event_test: Pass, RuntimeError # Issue #18931
polymer/test/entered_view_test: Pass, RuntimeError # Issue #18931
polymer/test/event_handlers_test: Pass, RuntimeError # Issue #18931
polymer/test/event_path_test: Pass, RuntimeError # Issue #18931
polymer/test/event_path_declarative_test: Pass, RuntimeError, Timeout # Issue #18931
polymer/test/events_test: Pass, RuntimeError # Issue #18931
polymer/test/instance_attrs_test: Pass, RuntimeError # Issue #18931
polymer/test/js_interop_test: Pass, RuntimeError # Issue #18931
polymer/test/nested_binding_test: Pass, RuntimeError # Issue #18931
polymer/test/noscript_test: Pass, RuntimeError # Issue #18931
polymer/test/prop_attr_bind_reflection_test: Pass, RuntimeError # Issue #18931
polymer/test/prop_attr_reflection_test: Pass, RuntimeError # Issue #18931
polymer/test/property_change_test: Pass, Timeout # 8931
polymer/test/publish_attributes_test: Pass, RuntimeError # Issue #18931
polymer/test/publish_inherited_properties_test: Pass, RuntimeError # Issue #18931
polymer/test/register_test: Pass, RuntimeError # Issue #18931
polymer/test/take_attributes_test: Pass, RuntimeError # Issue #18931
polymer/test/template_distribute_dynamic_test: Pass, RuntimeError # Issue #18931

@kasperl
Copy link

kasperl commented Jun 4, 2014

I assume this is something you want to try to fix for 1.5?


Removed Priority-Unassigned label.
Added Priority-High label.

@sigmundch
Copy link
Member

We have not been able to reproduce those flakes entirely.

They definitely don't fail in dart2js. We've done some things that we believe will help make it less likely to happen (I removed the use of the boot.js that we had at the time of the roll, and we are upgrading to the latest version of the polyfills).

I chatted with Rico about getting flaky history information, and unfortunately we don't have flaky logs for the Dartium bots, so he mentioned that the best we can do is remove the status lines and see what happens.

John plans to submit https://codereview.chromium.org/307793002 today, which will remove these pass/fail status lines.

@vsmenon
Copy link
Member Author

vsmenon commented Jun 6, 2014

John tried to remove but had to revert.

Let's try again after I roll to 36.0.1985.49 (https://code.google.com/p/dart/issues/detail?id=19269).

There are a few stability fixes there, include a couple related to html imports:
http://build.chromium.org/f/chromium/perf/dashboard/ui/changelog_blink.html?url=/branches/chromium/1985&range=173811:175393&mode=html

Do you think these might be related? I suspect our Polymer tests are the only ones using imports extensively.

@kasperl
Copy link

kasperl commented Jun 6, 2014

Adding Vijay as owner of this (under the assumption that the next step is to try to bring in the Blink stability fixes).


Set owner to @vsmenon.

@jmesserly
Copy link

ah yes, that could totally be the issue. Thank you Vijay! :D

@kasperl
Copy link

kasperl commented Jun 11, 2014

Any update on this?

@leafpetersen
Copy link
Member

I'm looking into this, I can reproduce locally. I'll update later today.

@vsmenon
Copy link
Member Author

vsmenon commented Jun 13, 2014

We've replicated the underlying issue (custom elements not upgrading at the expected time) with upstream Chrome (36 beta and 37 canary).

It only appears to occur if two frames are concurrently loading pages with custom elements - hence why we're seeing it in the test framework.

Dropping from this milestone. We still need a way to re-enable polymer tests - perhaps working around this bug if necessary.


Removed this from the 1.5 milestone.
Added this to the 1.6 milestone.

@sigmundch
Copy link
Member

because we don't use iframes with content_shell, we have been able to run polymer tests there. I can't recall exactly when we did the switch in the status files, but we have already enabled the tests in content_shell in bleeding_edge.

@vsmenon
Copy link
Member Author

vsmenon commented Aug 6, 2014

Removed this from the 1.6 milestone.
Added this to the 1.7 milestone.

@vsmenon
Copy link
Member Author

vsmenon commented Sep 16, 2014

Removed this from the 1.7 milestone.
Added this to the 1.8 milestone.

@whesse
Copy link
Member

whesse commented Nov 5, 2014

This test is flaky on dartium now:
polymer/example/component/news/test/news_index_test

It sounds like the same problem - the assertion that fails is
 expect(listComp is Polymer, true, reason: 'x-news should be created');

All the other tests in the list in the top comment remain marked as flaky in the status file, but this test was removed - it was marked failing before polymer 0.4, and removed in https://codereview.chromium.org//558673004. Restoring its flaky status.

@vsmenon
Copy link
Member Author

vsmenon commented Nov 5, 2014

Removed this from the 1.8 milestone.

@kasperl
Copy link

kasperl commented Nov 28, 2014

Added this to the 1.9 milestone.

@kasperl
Copy link

kasperl commented Nov 28, 2014

Added Accepted label.

@vsmenon
Copy link
Member Author

vsmenon commented Feb 17, 2015

Removed this from the 1.9 milestone.
Added this to the 1.10 milestone.

@vsmenon
Copy link
Member Author

vsmenon commented Apr 9, 2015

Is this still an issue?


Removed the owner.
Added Triaged label.

@sigmundch
Copy link
Member

It might be - we need to investigate. The flaky status is still in our .status file in polymer-dart here: https://github.com/dart-lang/polymer-dart/blob/master/.status

There was a bug creating the .flaky logs in the package-bots that was fixed this week, so we should be able to take another look.

@sigmundch
Copy link
Member

cc @jakemac53.

@kevmoo kevmoo added P2 A bug or feature request we're likely to work on type-bug Incorrect behavior (everything from a crash to more subtle misbehavior) and removed triaged labels Mar 1, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 A bug or feature request we're likely to work on type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)
Projects
None yet
Development

No branches or pull requests

8 participants