My favorites | Sign in
Project Home Downloads Wiki Issues Code Search
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 38497: CheckFalseTest.CheckFails and AssertionTest.Assertion are flaky because they don't wait properly
3 people starred this issue and may be notified of changes. Back to list
Status:  WontFix
Owner:  phajdan.jr@chromium.org
Closed:  Apr 2012
Cc:  andyb...@chromium.org

Restricted
  • Only users with Commit permission may comment.


Sign in to add a comment
 
Project Member Reported by thestig@chromium.org, Mar 18, 2010
Vista Tests (dbg)(3)

[----------] 1 test from CheckFalseTest
[ RUN      ] CheckFalseTest.CheckFails
C:\b\slave\chromium-dbg-builder\build\src\chrome\test\ui\ui_test.cc(198): 
error: Value of: actual_crashes
  Actual: 0
Expected: expected_crashes_
Which is: 1
Encountered an unexpected crash in the program during this test.  NOTE: 
This test is expected to fail if crash_service.exe is not running. Start it 
manually before running this test (see the build output directory).
[  FAILED  ] CheckFalseTest.CheckFails (2371 ms)

It seems to pass on other Windows bots, but not this one. Can you take a 
look? Sheriffs: if it's not fixed by the time you read this, please 
disable.
Mar 18, 2010
#1 maruel@chromium.org
Rebooting slave, we'll see.
Mar 18, 2010
#2 maruel@chromium.org
Incredible what a reboot can do.

Looking forward rebooting all the slaves once  issue 36843  is fixed.
Status: Fixed
Mar 18, 2010
#3 thestig@chromium.org
Sadly after the reboot, the second run started failing yet again. !#$@!
Status: Available
Mar 18, 2010
#4 bugdroid1@gmail.com
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=41970 

------------------------------------------------------------------------
r41970 | thestig@chromium.org | 2010-03-18 11:08:28 -0700 (Thu, 18 Mar 2010) | 5 lines
Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/common/logging_chrome_uitest.cc?r1=41970&r2=41969

Windows: Mark CheckFalseTest.CheckFails as flaky. It's failing on the Vista dbg bot.

BUG=38497
TEST=Vista dbg ui tests go green 100% of the time.
Review URL: http://codereview.chromium.org/1072003
------------------------------------------------------------------------

Apr 15, 2010
#5 maruel@chromium.org
(No comment was entered for this change.)
Owner: ---
Cc: -jh0763576
Labels: -Area-Undefined Area-BuildTools
Jul 21, 2010
#6 phajdan.jr@chromium.org
 Issue 49838  has been merged into this issue.
Cc: phajdan...@chromium.org thes...@chromium.org
Jul 21, 2010
#7 phajdan.jr@chromium.org
Okay, here's my theory:

1. The UI test infrastructure will shut down the browser as soon as the test body executes.
2. Here the test body is very short. It just sets a variable or two.
3. Moreover, the test decides not to wait for the browser to start up, so it proceeds extremely quickly.
4. So quickly, that we can exit faster than the crash is triggered.

Summary: the test failure means that we detected no crashes, and were expecting exactly one crash.

Lei, if it's going to take more time, could you update the status on Linux to FLAKY (it's now FAILS)? Please let me know if you'd need some help with infrastructure that would make possible making it solid.
Summary: CheckFalseTest.CheckFails is flaky because it doesn't wait properly
Status: Assigned
Owner: thes...@chromium.org
Cc: -phajdan...@chromium.org -thes...@chromium.org
Labels: -OS-Windows FlakyTest
Jul 21, 2010
#8 thestig@chromium.org
I looked at the Linux failure and determined it's related to PathService caching the old value for DIR_LOGS while DIR_USER_DATA changed. Unduping.
Status: Available
Owner: ---
Jul 30, 2010
#9 mal@google.com
(No comment was entered for this change.)
Labels: -FlakyTest Tests-Flaky
Mar 18, 2011
#10 lafo...@chromium.org
Vista Tests (dbg)(3)

[----------] 1 test from CheckFalseTest
[ RUN      ] CheckFalseTest.CheckFails
C:\b\slave\chromium-dbg-builder\build\src\chrome\test\ui\ui_test.cc(198): 
error: Value of: actual_crashes
  Actual: 0
Expected: expected_crashes_
Which is: 1
Encountered an unexpected crash in the program during this test.  NOTE: 
This test is expected to fail if crash_service.exe is not running. Start it 
manually before running this test (see the build output directory).
[  FAILED  ] CheckFalseTest.CheckFails (2371 ms)

It seems to pass on other Windows bots, but not this one. Can you take a 
look? Sheriffs: if it's not fixed by the time you read this, please 
disable.
Labels: -Area-BuildTools bulkmove Area-Build
May 26, 2011
#11 phajdan.jr@chromium.org
(No comment was entered for this change.)
Summary: CheckFalseTest.CheckFails and AssertionTest.Assertion are flaky because they don't wait properly
Status: Assigned
Owner: phajdan.jr@chromium.org
Sep 19, 2011
#12 bugdroid1@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=101760

------------------------------------------------------------------------
r101760 | phajdan.jr@chromium.org | Mon Sep 19 10:08:12 PDT 2011

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/automation/automation_provider_observers.h?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/content/browser/renderer_host/browser_render_process_host.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/ipc/ipc_channel_proxy.h?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/content/plugin/plugin_channel.h?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/test/automation/proxy_launcher.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/importer/firefox_importer_unittest_utils_mac.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/test/automation/proxy_launcher.h?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/common/logging_chrome_uitest.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/ipc/ipc_channel_proxy.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/automation/automation_provider_observers.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/content/plugin/plugin_thread.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/content/browser/child_process_launcher.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/content/browser/browser_child_process_host.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/content/common/gpu/gpu_channel_manager.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/test/automation/automation_proxy.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/content/common/gpu/gpu_channel.h?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/ipc/ipc_channel_posix.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/ipc/ipc_channel.h?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/ipc/ipc_channel_posix.h?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/content/common/gpu/gpu_channel.cc?r1=101760&r2=101759&pathrev=101760
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/test/automation/automation_proxy.h?r1=101760&r2=101759&pathrev=101760

Wait properly for renderer crashes

This replaces a Sleep in automation with a wait for renderer crash. It turns out that our IPC on POSIX had one loophole that caused it not to notice very early crashes, so I also fixed that.

The problem was that when the child process died before connecting to the parent's IPC channel, the parent wouldn't notice the crash because the child end of the IPC pipe was kept open for too long. This change makes the code close the child end of the pipe right after forking the child.

This might also help with automation not noticing the browser crash during initial launch, or at least should be a good step toward fixing that problem.

BUG=38497,90489

Review URL: http://codereview.chromium.org/7870008
------------------------------------------------------------------------
Sep 23, 2011
#13 maruel@google.com
http://build.chromium.org/p/tryserver.chromium/builders/linux_chromeos/builds/14079/steps/ui_tests/logs/stdio

[----------] 1 test from CheckFalseTest
[ RUN      ] CheckFalseTest.CheckFails
[22944:22944:4677598250:WARNING:zygote_host_linux.cc(149)] Running without the SUID sandbox! See https://code.google.com/p/chromium/wiki/LinuxSUIDSandboxDevelopment for more information on developing with the sandbox on.
[22944:22944:4677609266:ERROR:library_loader.cc(58)] Problem loading chromeos shared object: Couldn't load libcros from: /mnt/data/b/build/slave/linux_chromeos/build/src/out/Debug/chromeos/libcros.so error: /mnt/data/b/build/slave/linux_chromeos/build/src/out/Debug/chromeos/libcros.so: cannot open shared object file: No such file or directory
[22944:22944:4677609320:WARNING:cros_library.cc(66)] CrosLibrary failed to Initialize. Will use stub implementations.
[22944:22948:4677612675:ERROR:bus.cc(340)] Failed to get the ownership of org.chromium.LibCrosService: Connection ":1.1125" is not allowed to own the service "org.chromium.LibCrosService" due to security policies in the configuration file
[22944:22944:4677716562:WARNING:wm_ipc.cc(311)] Missing _CHROME_LAYOUT_MODE property on root window
[22944:22944:4677785277:ERROR:timezone_settings.cc(63)] GetTimezoneID: Cannot read timezone symlink /var/lib/timezone/localtime
[22944:22944:4677785358:ERROR:timezone_settings.cc(148)] Got an empty string for timezone, default to America/Los_Angeles
[22944:22944:4677794479:WARNING:input_method_manager.cc(223)] No active input methods found.
[22944:22944:4677794695:WARNING:input_method_manager.cc(223)] No active input methods found.
[22944:22944:4677794951:WARNING:input_method_manager.cc(223)] No active input methods found.
[22944:22949:4677825781:WARNING:touchpad_settings.cc(26)] /opt/google/touchpad/tpcontrol not found. Maybe running on Linux desktop?
[22944:22949:4677825896:WARNING:touchpad_settings.cc(26)] /opt/google/touchpad/tpcontrol not found. Maybe running on Linux desktop?
[22944:22944:4677826077:ERROR:input_method_manager.cc(830)] Failed to change keyboard layout to us
[22944:22944:4677831253:ERROR:proxy_resolution_service_provider.cc(216)] Failed to export org.chromium.LibCrosServiceInterface.ResolveNetworkProxy
[22944:22944:4677908785:WARNING:native_widget_gtk.cc(419)] compositing not supported; allowing anyway
[22966:22966:4677815211:FATAL:renderer_main.cc(84)] Check failed: false. 
Backtrace:
	base::debug::StackTrace::StackTrace() [0x7fa3352cd186]
	logging::LogMessage::~LogMessage() [0x7fa3352f23f6]
	HandleRendererErrorTestParameters() [0x7fa33789383d]
	RendererMain() [0x7fa337893ac2]
	(anonymous namespace)::RunZygote() [0x7fa3351fa76e]
	(anonymous namespace)::RunNamedProcessTypeMain() [0x7fa3351fa937]
	content::ContentMain() [0x7fa3351faedc]
	ChromeMain [0x7fa33423ae68]
	main [0x7fa334239a7c]
	0x7fa32cc56c4d
	0x7fa334239989

[22944:22954:4677970026:WARNING:file_descriptor_set_posix.cc(21)] FileDescriptorSet destroyed with unconsumed descriptors
[22944:22954:4677970138:WARNING:file_descriptor_set_posix.cc(21)] FileDescriptorSet destroyed with unconsumed descriptors
[22944:22954:4677970214:WARNING:file_descriptor_set_posix.cc(21)] FileDescriptorSet destroyed with unconsumed descriptors
Entity: line 11: parser error : Extra content at the end of the document
 <window action="destroy" windowid="0" parent="0" session="0" time="1316756766"/
                                                                               ^
error : xmlTextWriterWriteDocCallback : XML error 5 !
chrome/test/ui/ui_test.cc:154: Failure
Value of: assertions.size()
  Actual: 2
Expected: expected_errors_
Which is: 1
The following error(s) occurred in the application during this test:

[22940:22940:4677100652:FATAL:renderer_main.cc(77)] Check failed: false. 

[22966:22966:4677815211:FATAL:renderer_main.cc(84)] Check failed: false. 
[  FAILED  ] CheckFalseTest.CheckFails (715 ms)
[----------] 1 test from CheckFalseTest (715 ms total)

Nov 8, 2011
#14 dhar...@google.com
(No comment was entered for this change.)
Labels: Hotlist-Fixit
Jan 6, 2012
#15 msw@chromium.org
CheckFalseTest.CheckFails is failing pretty consistently on the official linux32|64 bots.
Please take a look at your earliest convenience.
http://chrome-master2.mtv:8011/builders/linux64%20trunk/builds/2740
http://chrome-master2.mtv:8011/builders/linux32%20trunk/builds/2641

CheckFalseTest.CheckFails: 
[11454:11457:5364731054416:ERROR:object_proxy.cc(239)] Failed to call method: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.NetworkManager was not provided by any .service files
chrome/test/ui/ui_test.cc:153: Failure
Value of: assertions.size()
  Actual: 0
Expected: expected_errors_
Which is: 1
The following error(s) occurred in the application during this test:
Feb 15, 2012
#16 bugdroid1@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=122137

------------------------------------------------------------------------
r122137 | evan@chromium.org | Wed Feb 15 12:23:57 PST 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/renderer/external_extension_uitest.cc?r1=122137&r2=122136&pathrev=122137
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/installer/util/product_unittest.cc?r1=122137&r2=122136&pathrev=122137
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/ui/cocoa/applescript/browsercrapplication+applescript_test.mm?r1=122137&r2=122136&pathrev=122137
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/common/logging_chrome_uitest.cc?r1=122137&r2=122136&pathrev=122137
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/common/service_process_util_unittest.cc?r1=122137&r2=122136&pathrev=122137
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/ui/cocoa/browser/avatar_button_controller_unittest.mm?r1=122137&r2=122136&pathrev=122137
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/ui/cocoa/view_id_util_browsertest.mm?r1=122137&r2=122136&pathrev=122137
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/installer/util/copy_tree_work_item_unittest.cc?r1=122137&r2=122136&pathrev=122137
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/service/cloud_print/cloud_print_url_fetcher_unittest.cc?r1=122137&r2=122136&pathrev=122137
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/common/important_file_writer_unittest.cc?r1=122137&r2=122136&pathrev=122137

Flakiness cleanup: disable flaky tests under chrome/

BUG=84299,99469,90557,109292,38497,45243,84854,59785,59783,59784,100567,62777,60426
TBR=thakis

Review URL: https://chromiumcodereview.appspot.com/9406017
------------------------------------------------------------------------
Apr 18, 2012
#17 jam@chromium.org
these tests are now removed
Status: WontFix
Oct 13, 2012
#18 bugdroid1@chromium.org
This issue has been closed for some time. No one will pay attention to new comments.
If you are seeing this bug or have new data, please click New Issue to start a new bug.
Labels: Restrict-AddIssueComment-Commit
Mar 9, 2013
#19 bugdroid1@chromium.org
(No comment was entered for this change.)
Labels: -Tests-Flaky -Area-Build Cr-Tests-Flaky Build
Sign in to add a comment

Powered by Google Project Hosting