My favorites | Sign in
Project Home Downloads Wiki Issues Code Search
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 157246: login timeout with net::ERR_ABORTED failures
10 people starred this issue and may be notified of changes. Back to list
 
Reported by be...@google.com, Oct 15, 2012
The test tries to login for real (i.e without a mocked auth server), dont see any immediate reason it should fail other than recent login flakes.

screenshot shows we're stuck on the credentials screen: http://atlantis4.mtv.corp.google.com/results/796721-chromeos-test/chromeos1-rack5-host7/login_RemoteLogin/results/login-timeout-fail-screenshot-0.png



Oct 15, 2012
#1 be...@chromium.org
(No comment was entered for this change.)
Labels: -Area- Area-Network
Oct 15, 2012
#2 dd...@chromium.org
This is causing BVT failures intermittently across R23 & R24 on multiple platforms. 
Status: Assigned
Owner: tr...@chromium.org
Cc: r...@chromium.org
Labels: -Pri-2 -Sev- Pri-1 Sev-1 Mstone-23 Mstone-24 bvt
Oct 16, 2012
#3 be...@google.com
observed similar failures in a another test which tries real login: http://atlantis4.mtv.corp.google.com/results/799486-chromeos-test/chromeos1-rack5-host5/desktopui_DocViewing/
Oct 17, 2012
#4 be...@google.com
these test try real logins, and it looks like 50% of the time they fail because of a login timeout (the other times its for a bunch of different reasons): login_RemoteLogin, desktopui_DocViewing, desktopui_NaClSanity.

http://atlantis4.mtv.corp.google.com/results/803253-chromeos-test/chromeos-rack8e-host4/desktopui_NaClSanity/results/

Oct 17, 2012
#5 craigdh@chromium.org
 Issue 34113  has been merged into this issue.
Cc: tbroch@chromium.org dpolukhin@chromium.org rginda@chromium.org ygorshenin@chromium.org nkostylev@chromium.org hashimoto@chromium.org
Oct 17, 2012
#6 craigdh@chromium.org
(No comment was entered for this change.)
Labels: Test-Support Test-Followup
Oct 17, 2012
#7 craigdh@chromium.org
(No comment was entered for this change.)
Labels: -Test-Support
Oct 17, 2012
#8 dd...@chromium.org
Talked with beeps@, marking this as a dev blocker. This has been failing pretty regularly on zgb for R24
Labels: ReleaseBlock-Dev
Oct 17, 2012
#9 dd...@chromium.org
(No comment was entered for this change.)
Labels: -Mstone-23
Oct 17, 2012
#10 be...@google.com
running login_RemoteLogin and desktopui_NaClSanity 100 times in oysterbay. If these pass it might indicate a Atlantis networking issue.
Oct 17, 2012
#11 scottz@chromium.org
The device you listed there, chromeos-rack8e-host4, is not in Atlantis it is in Oyster Bay. 

The name "atlantis4" in the URL is referring to the server that is storing the results itself. This system is on a different network than the currently affected lab network in Atlantis. 

Oct 17, 2012
#12 pstew@chromium.org
I've tried to repro this as well and have failed.  Because you continue to have remote control of the client (including retrieving screenshots of the device in its failed state), I have to assume this is in fact not a network issue.
Oct 17, 2012
#13 pstew@chromium.org
[Not a network issue local to the client.]
Oct 17, 2012
#14 tr...@google.com
+xiaowenx, scottz since it looks to involve lab setup
Owner: scottz@chromium.org
Cc: xiaow...@chromium.org
Oct 17, 2012
#15 scottz@chromium.org
This is not a lab issue. 

Beeps was looking at reproducing this, beeps were you able to reproduce this using the same image? 
Cc: craigdh@chromium.org
Oct 17, 2012
#16 be...@chromium.org
(No comment was entered for this change.)
Owner: be...@chromium.org
Oct 17, 2012
#17 be...@chromium.org
I ran login_Remote 50 times on actual hardware, locally (a lumpy imaged with 3055.0.0) and it passed. The only issue i observed was a sig 11 from: https://code.google.com/p/chromium/issues/detail?id=156466

Reassigning to trond.
Owner: tr...@chromium.org
Oct 17, 2012
#18 be...@chromium.org
the sig 11 was from desktopui_DocViewing, with which I didn't observe any login issues either.
Oct 17, 2012
#19 tr...@chromium.org
(No comment was entered for this change.)
Status: Fixed
Oct 18, 2012
#20 johndh...@chromium.org
desktopui_NaClSanity failed and the host invovled is located in Oyster Bay

http://chromegw/i/chromiumos.release/builders/x86-zgb%20full%20release-R23-2913.B/builds/61

http://chromegw/i/chromiumos.release/builders/x86-zgb%20full%20release-R23-2913.B/builds/61/steps/HWTest%20%5Bbvt%5D%20%5Bx86-zgb%5D/logs/stdio


Screenshot
http://atlantis4.mtv.corp.google.com/results/810241-chromeos-test/chromeos1-rack6-host8/desktopui_NaClSanity/results/login-timeout-fail-screenshot-0.png


Job
http://cautotest/afe/#tab_id=view_job&object_id=810241
Summary: login_RemoteLogin, desktopui_NaClSanity fails on login timeout
Status: Assigned
Oct 18, 2012
#21 cmasone@chromium.org
I have several questions

1) Why is any test, beyond login_RemoteLogin (which we shouldn't be running in my opinion) doing a login against the live internet?
2) Why do we think this has anything to do with our network stack, when it is far more likely that it's a Chrome or test automation problem?

Beeps, if you're the owner of the PyAutoFunctionalTests stuff now, it's up to you to capture the information that will allow us to diagnose these hard-to-reproduce failures.
Owner: be...@chromium.org
Labels: -Area-Network Area-Test
Oct 18, 2012
#22 be...@chromium.org
1) Why is any test, beyond login_RemoteLogin (which we shouldn't be running in my opinion) doing a login against the live internet?

I guess because it inherits from pyui, and that class never set up an auth server. Although even if we set up an auth server and these started passing wouldn't there still be an issue?

2) Why do we think this has anything to do with our network stack, when it is far more likely that it's a Chrome or test automation problem?

I don't think it's test automation because login_Remote really does nothing else except login, and the other tests that inherit from cros_ui seem ok. I'm trying to repro on zgb right now.
Oct 18, 2012
#23 johndh...@chromium.org
ZGB
login_RemoteLogin failed and the host is in Oyster Bay as well.
R22-2723.167.0

http://chromegw/i/chromiumos.release/builders/x86-zgb%20full%20release-R22-2723.B/builds/145

http://chromegw/i/chromiumos.release/builders/x86-zgb%20full%20release-R22-2723.B/builds/145/steps/HWTest%20%5Bbvt%5D%20%5Bx86-zgb%5D/logs/stdio


Screenshot
http://atlantis4.mtv.corp.google.com/results/809932-chromeos-test/chromeos1-rack6-host3/login_RemoteLogin/results/login-timeout-fail-screenshot-0.png

Job
http://cautotest/afe/#tab_id=view_job&object_id=809932
Oct 18, 2012
#24 be...@google.com
craighs adding an auth server while I try and get a repo:
https://code.google.com/p/chromium-os/issues/detail?id=35461&thanks=35461&ts=1350583228
Oct 18, 2012
#25 johndh...@chromium.org
ZGB
login_RemoteLogin, experimental_enterprise_DevicePolicy, experimental_desktopui_NaClSanity
R24-3059.0.0

http://chromegw/i/chromeos/builders/x86-zgb%20canary/builds/2506

http://chromegw/i/chromeos/builders/x86-zgb%20canary/builds/2506/steps/HWTest%20%5Bbvt%5D%20%5Bx86-zgb%5D/logs/stdio


Screenshots
http://atlantis4.mtv.corp.google.com/results/810765-chromeos-test/chromeos1-rack6-host3/login_RemoteLogin/results/login-timeout-fail-screenshot-0.png

http://atlantis4.mtv.corp.google.com/results/810795-chromeos-test/chromeos1-rack6-host3/desktopui_NaClSanity/results/login-timeout-fail-screenshot-0.png

http://atlantis4.mtv.corp.google.com/results/810792-chromeos-test/chromeos1-rack6-host4/enterprise_DevicePolicy/results/login-timeout-fail-screenshot-0.png

Jobs
http://cautotest/afe/#tab_id=view_job&object_id=810765
http://cautotest/afe/#tab_id=view_job&object_id=810792
http://cautotest/afe/#tab_id=view_job&object_id=810795
Oct 18, 2012
#26 cmasone@chromium.org
If it was our network stack, we'd hear reports of this hang in the wild.

Please make login_RemoteLogin experimental for now.

Instead of continuing to try to repro right now, can you investigate ways to get the info we need on the bots, where this happens all the time?  And then work to get a repro?
Oct 18, 2012
#27 craigdh@chromium.org
fyi, I have a cl in flight to switch desktopui_DocViewing, desktopui_NaClSanity, and other PyAuto-based tests to use the fake authserver.
Oct 18, 2012
#28 dd...@chromium.org
Re: comment # 26, R24 isn't rolled out to very many users and isn't on Dev.
If this problem is specific to R24, then that's not necessarily a valid
assumption.
Oct 18, 2012
#29 be...@chromium.org
marked login_RemoteLogin experimental https://gerrit.chromium.org/gerrit/#/c/35988/
the others should be fixed with craighs cl
Oct 18, 2012
#30 cmasone@chromium.org
beeps, there's no guarantee that craig's change will do anything.  What're your plans for getting the information you need to diagnose this problem?
Oct 18, 2012
#31 be...@chromium.org
From the pyauto side it really doesn't do anything for login except check in a queue for success/failure events. most of the time it just times out waiting for an event, so we need a way to account for what was going on in those 45 seconds?

I could try and use autotest profiler, ive been looking at tcp dumps in comparison to successful logins. another option is to add more descriptive log statements to the entire login flow. what do you thing? should we talk about this offline?

we could also try something like pinging if a login fails, to confirm that the network isn't totally down.
Oct 18, 2012
#32 cmasone@chromium.org
Looking at the test runs that John posted above, from R22 and R24, the chrome logs from these tests has the following line in it:

[3511:3511:183997730:WARNING:parallel_authenticator.cc(380)] Login failed: net::ERR_ABORTED

Per the Chrome source, this means "An operation was aborted (due to user action)."

Here are the links to the results for these tests; can someone who can grok the log output from the auth extension please look at this?

R22 test:
http://atlantis4.mtv.corp.google.com/results/809932-chromeos-test/chromeos1-rack6-host3/login_RemoteLogin/results/

R24 tests:
http://atlantis4.mtv.corp.google.com/results/810765-chromeos-test/chromeos1-rack6-host3/login_RemoteLogin/results/
http://atlantis4.mtv.corp.google.com/results/810792-chromeos-test/chromeos1-rack6-host4/enterprise_DevicePolicy/results/
http://atlantis4.mtv.corp.google.com/results/810795-chromeos-test/chromeos1-rack6-host3/desktopui_NaClSanity/results/
Oct 18, 2012
#33 cmasone@chromium.org
And I did check; that did not appear in a log from a successful test.
Oct 18, 2012
#34 be...@google.com
Looked through the chrome logs, In the successfull case it looks like it never enters NetworkChangeNotifierChromeos::UpdateConnectivityState, instead directly giving us:
[4352:4352:1018/073234:INFO:CONSOLE(6163)] "GaiaSigninScreen.onMessage_: method=loginUILoaded", source: chrome://oobe/login (6163)
[4352:4352:1018/073234:INFO:CONSOLE(6568)] "updateState_: state=1, network=Ethernet, reason=update, lastNetworkType=1, opt_forceUpdate=undefined", source: chrome://oobe/login (6568)
[4352:4352:1018/073234:INFO:CONSOLE(6163)] "GaiaSigninScreen.onMessage_: method=completeLogin", source: chrome://oobe/login (6163)

http://atlantis4.mtv.corp.google.com/results/810764-chromeos-test/chromeos1-rack6-host4/login_LoginSuccess/results/
or stopping at UpdateNetworkState:
http://atlantis4.mtv.corp.google.com/results/806267-chromeos-test/chromeos2-row1-rack5-host5/login_RemoteLogin/results/

As opposed to the fail case, where we see multiple lines from the network_change_notifier's UpdateConnectivityState:
[3720:3720:1018/073708:VERBOSE1:network_change_notifier_chromeos.cc(175)] UpdateConnectivityState: Ethernet, type= 1, device= /device/eth0, state= 10, connect= 0, type= 6
http://atlantis4.mtv.corp.google.com/results/810765-chromeos-test/chromeos1-rack6-host3/login_RemoteLogin/results/

Following the code a little I think it went through the extension at screen_gaia_signin.js which eventually delegates to ParallelAuthenticator::CompleteLogin through the webui, with a failure. 
Would be great if someone more qualified could shed some light.
Oct 19, 2012
#35 be...@chromium.org
assigning to nikita to get a better idea of why/if the extension failed.
Owner: nkostylev@chromium.org
Oct 22, 2012
#36 kr...@chromium.org
 Issue 35547  has been merged into this issue.
Oct 22, 2012
#37 dd...@chromium.org
(No comment was entered for this change.)
Labels: OS-Chrome
Oct 22, 2012
#38 dd...@chromium.org
(No comment was entered for this change.)
Labels: -ReleaseBlock-Dev ReleaseBlock-Beta
Oct 23, 2012
#39 dennisjeffrey@chromium.org
Happened again on login_RemoteLogin for "x86-zgb-r22 build, R22-2723.177.0, running on zgb".

10/23 03:29:44.324 DEBUG|site_autot:0098| AUTOTEST_STATUS::START	----	----	timestamp=1350988184	localtime=Oct 23 03:29:44	
10/23 03:29:44.325 INFO |server_job:0120| START	----	----	timestamp=1350988184	localtime=Oct 23 03:29:44	
10/23 03:29:44.572 DEBUG|site_autot:0098| AUTOTEST_STATUS::	START	login_RemoteLogin	login_RemoteLogin	timestamp=1350988184	localtime=Oct 23 03:29:44	
10/23 03:29:44.573 INFO |server_job:0120| 	START	login_RemoteLogin	login_RemoteLogin	timestamp=1350988184	localtime=Oct 23 03:29:44	
10/23 03:30:31.222 DEBUG|site_autot:0098| AUTOTEST_STATUS::		FAIL	login_RemoteLogin	login_RemoteLogin	timestamp=1350988231	localtime=Oct 23 03:30:31	Login timed out. Please check the file named  login-timeout-fail-screenshot.png located in the results folder.
10/23 03:30:32.653 INFO |server_job:0120| 		FAIL	login_RemoteLogin	login_RemoteLogin	timestamp=1350988231	localtime=Oct 23 03:30:31	Login timed out. Please check the file named  login-timeout-fail-screenshot.png located in the results folder.
10/23 03:30:32.659 DEBUG|site_autot:0098| AUTOTEST_STATUS::	END FAIL	login_RemoteLogin	login_RemoteLogin	timestamp=1350988231	localtime=Oct 23 03:30:31	
10/23 03:30:32.659 INFO |server_job:0120| 	END FAIL	login_RemoteLogin	login_RemoteLogin	timestamp=1350988231	localtime=Oct 23 03:30:31	
Oct 23, 2012
#40 dennisjeffrey@chromium.org
Happened again for login_RemoteLogin on "x86-alex-r24 build, R24-3083.0.0, running on alex".  I'm attaching the fail screenshot -- seems like the credentials are added in the login screen.
login-timeout-fail-screenshot-0.png
952 KB   View   Download
Oct 23, 2012
#41 nkostylev@chromium.org
Comments #39/#40 show that there's something wrong with test infrastructure.
As you could see real GAIA frame is loaded instead of fake auth page.

Fake DNS resolver / local auth server issue?
Status: Available
Owner: craigdh@chromium.org
Oct 23, 2012
#42 nkostylev@chromium.org
(No comment was entered for this change.)
Summary: login_RemoteLogin, desktopui_NaClSanity fails on login timeout (real GAIA frame is loaded instead of fake one)
Oct 23, 2012
#43 craigdh@chromium.org
The title of this bug was misleading, I've fixed it.

The whole point of login_RemoteLogin is that it tries a live login. This live login is what is failing.

The issue with desktopui_NaCLSanity has been resolved. It was seeing the same problem because was not previously using the fake authserver, which it has now been switch to use. No issues are seen logging in with the fake authserver.
Summary: login_RemoteLogin fails on login timeout
Owner: nkostylev@chromium.org
Oct 23, 2012
#44 nkostylev@chromium.org
Thanks for clarification, I'll investigate.
Oct 24, 2012
#45 dennisjeffrey@chromium.org
Happened in bvt on "x86-alex-r23 build, R23-2913.134.0, running on alex".

Failure screenshot (attached) is different here than in comment 40.
login-timeout-fail-screenshot-0.png
19.0 KB   View   Download
Oct 25, 2012
#46 nkostylev@chromium.org
Yuri, please help investigate. Need to make sure that this is not caused by recent refactoring changes in this area.
Status: Assigned
Owner: ygorshenin@chromium.org
Cc: -ygorshenin@chromium.org
Labels: Iteration-67
Oct 26, 2012
#47 dennisjeffrey@chromium.org
Happened again in bvt on:

"x86-zgb-r22 build, R22-2723.185.0, running on zgb"

Failing screenshot attached.
login-timeout-fail-screenshot-0.png
46.2 KB   View   Download
Oct 26, 2012
#48 craigdh@chromium.org
Do we have any recent instances of this on ToT?
Oct 26, 2012
#49 dd...@chromium.org
It had been happening a few days ago on ToT but seems to have stopped.
Oct 26, 2012
#50 mlc...@chromium.org
M24 --> M25
Labels: -Mstone-24 MovedFromM24 Mstone-25
Oct 26, 2012
#51 dennisjeffrey@chromium.org
If this is indeed fixed on ToT, should we merge the fix into prior branches? Comments 45 and 47 show that it still periodically happens on R22 and R23.
Oct 29, 2012
#52 nkostylev@chromium.org
>> should we merge the fix into prior branches?

I'm not sure that it would be easy to identify a fix.
Oct 29, 2012
#53 nkostylev@chromium.org
(No comment was entered for this change.)
Labels: Iteration-68
Oct 31, 2012
#54 be...@chromium.org
Reactivating, just saw another failure. Looking back at all the chrome logs in this issue they seem laced with net::ERR as described in 35926:
 
http://atlantis4.mtv.corp.google.com/results/910879-chromeos-test/chromeos1-rack6-host3/login_RemoteLogin/

changed the title and merged 35926 into this. please take a look and confirm eitherway. 
Summary: login timeout with net::ERR_ABORTED failures
Oct 31, 2012
#55 be...@chromium.org
 Issue chromium-os:35926  has been merged into this issue.
Cc: avayvod@chromium.org altimof...@chromium.org dil...@chromium.org glotov@chromium.org davemo...@chromium.org ivankr@chromium.org dennisjeffrey@chromium.org
Nov 1, 2012
#57 kr...@chromium.org
@comment 52, my concern is we will be riding 23 for a while and will continue to hit this as we release builds.

Do we know what the effort would be to get this into 23?
Nov 1, 2012
#58 dd...@chromium.org
It can be considered for merge back once we have the fix running on ToT for a while. But we have to know the fix first.
Nov 1, 2012
#59 nkostylev@chromium.org
(No comment was entered for this change.)
Cc: -dpolukhin@chromium.org -rginda@chromium.org -avayvod@chromium.org -altimof...@chromium.org -dil...@chromium.org -glotov@chromium.org -davemo...@chromium.org
Nov 1, 2012
#60 nkostylev@chromium.org
I think it would be quite hard to identify and merge the fix to M23 since ToT changed in this area - refactoring, several fixes. Plus we don't have resources for now - planning to fix all M23 regression by the end of this week and switch to M25 feature development.

Are we sure that this doesn't show up for M25?
Nov 1, 2012
#61 be...@chromium.org
We still see login failures with net::ERR_ABORTED messages, not as frequently as of a week ago though. This might indicate that some issues were fixed. The failure in #56 occured on R25.
Nov 12, 2012
#65 nkostylev@chromium.org
(No comment was entered for this change.)
Labels: Iteration-69
Nov 12, 2012
#66 ygorshenin@chromium.org
beeps, could you please re-post logs you posted in #63. They're not available now.
Also, logs you posted in #64 are for enterprise_DevicePolicy autotest. How it's related to login_RemoveLogin?

I run 100 times login_RemoteLogin on lumpy and didn't notice any failures.

Nov 12, 2012
#67 be...@chromium.org
these login failures are not specific to login_RemoteLogin, general login failures have been hitting the queue for about two weeks. I went through the failure logs and compared them to the successful ones, and created bugs for messages that only occured in  the former. this bug is for tracking when we fail to login with net::ERR_ABORTED in the chrome logs, like in #64: 

https://doc-1c3qk-1i36k-s-googleusercontent.commondatastorage.googleapis.com/gs/8qmnhp46pk4tu8k1ednpcailb8kahpks/trmkc46kka660bq7j8hm4rt1qt32sqjj/1352736000000/chromeos-autotest-results/13747286673071246342/ChljaHJvbWVvcy1hdXRvdGVzdC1yZXN1bHRzEmE5Njk5OTItY2hyb21lb3MtdGVzdC9jaHJvbWVvczEtcmFjazUtaG9zdDUvZW50ZXJwcmlzZV9EZXZpY2VQb2xpY3kvcmVzdWx0cy9jaHJvbWVfMjAxMjExMDktMDAwMTU1?a=AGjQbs7LsGLDkVnPByhGgeJP_cYjqNK7cswGt-TFhp0CSFJEFUlQmMUNpUBecO-FyYsexgmPm0a_JTihwGBL-77aTl_6YRm73foCvmtxiZt8hp75wE1ZQ_9sp0sGRRN6GHCeTtyqXjxRGgryCsC_5dAWliBWu3veMYSTjbhyfIvpXtZAcknCfhtbtMTTsqgKoCYrtEviK0dEtR2ryO6VG067iQa1AUqjZoq3BXlJfvUnWRlV6tkqqnL3u_WzQqQ7tlu198yjHZgnu0G5l2B9z5Yl1ZewlM67Nlonf30p201xtgxTNWCOe5krLKD8cvvzkC2hnXZulxtCRIUPH7gEN2a0t71qfw36tH9RfIWB8aIDw7DmyXJ_bF4_TdTMQzeK7zKA6rXkGMCjcafGF1YvuK5S-oX5ryZ9YekEOAPd0Vr9LKP6o6nrLH4okt136qQAOd6Xn958mhgXrLhUrs8tr0WoLpUohidW2lu8qwIn9az_PcrbV6BaQB0tvf_u6dHrslAyvsihD2ybf3ZimZeca5Wk3PAYAyr82LfBhUbMrVyDjkWsNJjZNDvRHMi8l7h7ofLrSWTRn7M7eGrz0uuuwsGX7lM835HI17cMWLhsXnTEFWZRbsle4FPJizcg2yugaHCV_417l3axgNPmSTSvhGa0sAjzKmwntdZRAOPBG0jHyrrv4UQ-v3PAHqgKyvDSgbUzgw9Bp5UqvP2JQUC1ClhWdhUurydAJw&nonce=lfg97cf7ro5aq&user=13747286673071246342&hash=jjfv86ua2b2ef4l8hhir3h6btmkmf20s

There are a couple of other related bugs: 
https://code.google.com/p/chromium-os/issues/detail?id=35927&thanks=35927&ts=1351714188
https://code.google.com/p/chromium/issues/detail?id=158775


I have had bad luck with trying to repro it locally too. I believe dennisjeffrey is the sheriff today, he will probably be your best contact if this happens again. It happens on and off, so I will send an email to the lab infrastructure devs in an effort to get something you can analyze.

Nov 12, 2012
#68 scottz@chromium.org
Are there more things we can log to determine where and potentially why this is failing? 
Nov 12, 2012
#69 be...@chromium.org
from the pyauto side we usually just time out waiting for news about the login request. 
Nov 13, 2012
#70 dennisjeffrey@chromium.org
Login timeout happened again on 2 consecutive bvt runs on daisy: R25-3194.0.0 and R25-3196.0.0.

Both fail screenshots are the same - attached.
login-timeout-fail-screenshot-0.png
1013 KB   View   Download
Nov 13, 2012
#71 nkostylev@chromium.org
>> Both fail screenshots are the same - attached.

Please include links to logs each time.
It is a bit hard to keep track of those buildbot runs.
Nov 14, 2012
#72 ygorshenin@chromium.org
beeps: I receives "Access is denied" message while trying to access logs in #67.
Nov 14, 2012
#75 be...@chromium.org
sorry about that, looks like linking the exact log file doesn't seem to work for some reason. The log Im talking about can be found in the chrome log of the results folder of:
https://storage.cloud.google.com/?arg=chromeos-autotest-results/969992-chromeos-test/chromeos1-rack5-host5/enterprise_DevicePolicy#chromeos-autotest-results%2F969992-chromeos-test%2Fchromeos1-rack5-host5%2Fenterprise_DevicePolicy

you'll see something like:
[8522:8522:1109/000203:VERBOSE2:network_change_notifier_chromeos.cc(209)]  UpdateConnectivityState4: new_cs = 10, end_cs_ = 10prev_type = 6, new_type_ = 1
[8522:8522:1109/000203:WARNING:parallel_authenticator.cc(364)] Login failed: net::ERR_ABORTED
[8522:8522:1109/000203:VERBOSE2:automation_event_queue.cc(80)] AutomationEventQueue::NotifyEvent id=0
[8522:8522:1109/000203:VERBOSE2:automation_event_queue.cc(94)] AutomationEventQueue::RemoveObserver id=0
Nov 26, 2012
#76 ygorshenin@chromium.org
(No comment was entered for this change.)
Status: Started
Nov 27, 2012
#77 nkostylev@chromium.org
(No comment was entered for this change.)
Labels: Iteration-70
Nov 30, 2012
#78 ygorshenin@chromium.org
(No comment was entered for this change.)
Status: Assigned
Owner: nkostylev@chromium.org
Nov 30, 2012
#79 nkostylev@chromium.org
Thanks to Yuri's analysis we've got more understanding of 2 types of errors that are happening:

1. Cryptohome reports that it could not be mounted for unknown reason. That seems to be very strange since test creates brand new cryptohome.
Yuri will add more logging to get detailed cryptohome error for that case.
Message in logs "Cryptohome could not be mounted".

2. Login failed: net::ERR_ABORTED
This error should not be happening for existing code as in case of new user sign in authenticator should only create cryptohome and nothing more.
Here we have an example of cryptohome mount/create code somehow relies on network state which sounds like old (supposed to be unused) legacy code comes into play.
I'll do the cleanup.
Nov 30, 2012
#80 nkostylev@chromium.org
Not a Beta blocker. Seems to be happening only in test environment or/and in rare cases.
Labels: -ReleaseBlock-Beta
Dec 5, 2012
#81 be...@chromium.org
on a daisy-r25 build, R25-3345.0.0. Logs:
https://storage.cloud.google.com/?arg=chromeos-autotest-results/1155828-chromeos-test/chromeos2-row5-rack2-host8/login_CryptohomeMounted#chromeos-autotest-results%2F1155828-chromeos-test%2Fchromeos2-row5-rack2-host8%2Flogin_CryptohomeMounted

doesn't seem to have either of the symptoms in #79, is this the same type of login flake we've been seeing all along? I looked through sys and test logs briefly, but am prepared to dig deeper if you feel it'll help. 


Dec 5, 2012
#82 nkostylev@chromium.org
Now that we've lost INFO level log messages from JS console it is hard to understand what was happening in recent failure mentioned in comment #81.
Dec 10, 2012
#83 nkostylev@chromium.org
Tracking work item to get that useful info back in logs in  issue 165113 .
Dec 10, 2012
#84 nkostylev@chromium.org
(No comment was entered for this change.)
Labels: Iteration-71
Dec 14, 2012
#85 nkostylev@chromium.org
(No comment was entered for this change.)
Labels: -Mstone-25 Mstone-26
Dec 24, 2012
#86 nkostylev@chromium.org
Iteration-71 -> 72
Labels: Iteration-72
Dec 26, 2012
#87 be...@chromium.org
login timeout on parrot-r26 build, R26-3455.0.0, logs:
https://storage.cloud.google.com/?arg=chromeos-autotest-results/1286638-chromeos-test/chromeos2-row3-rack7-host8/login_LoginSuccess#chromeos-autotest-results%2F1286638-chromeos-test%2Fchromeos2-row3-rack7-host8%2Flogin_LoginSuccess

	START	login_LoginSuccess	login_LoginSuccess	timestamp=1356229423	localtime=Dec 22 18:23:43	
		FAIL	login_LoginSuccess	login_LoginSuccess	timestamp=1356229593	localtime=Dec 22 18:26:33	Login timed out. Please check the file named  login-timeout-fail-screenshot.png located in the results folder.
	END FAIL	login_LoginSuccess	login_LoginSuccess	timestamp=1356229593	localtime=Dec 22 18:26:33	
Dec 27, 2012
#88 nkostylev@chromium.org
Comment #87 fail screenshot shows that test auth form has been loaded.
Jan 9, 2013
#89 be...@chromium.org
Does that mean it's a different issue? 
This happens from time to time in the BVTs, not nearly as often as before but they're still login failures. Should we continue posting to this bug or split into more specific bugs?

1. parrot-r26 build, R26-3521.0.0, Logs:
http://atlantis2.mtv.corp.google.com/results/1415243-chromeos-test/chromeos2-row3-rack5-host9/login_CryptohomeMounted/

2. kiev-r26 build, R26-3523.0.0, Logs:
http://atlantis4.mtv.corp.google.com/results/1418101-chromeos-test/chromeos2-row5-rack7-host1/security_ProfilePermissions.login/

Both screenshots show that the test auth form has been loaded.
Jan 11, 2013
#90 nkostylev@chromium.org
Re: comment #89.

Yes, that seems to be a different issue.
This issue should track only those errors that have net::ERR_ABORTED failures in logs.
Jan 11, 2013
#91 nkostylev@chromium.org
(No comment was entered for this change.)
Labels: LoginUserMgmt-Fixit
Feb 12, 2013
#92 nkostylev@chromium.org
(No comment was entered for this change.)
Labels: -Mstone-26 Mstone-27
Mar 10, 2013
#93 bugdro...@chromium.org
(No comment was entered for this change.)
Labels: -Area-Test -Mstone-27 M-27 Cr-Test
Apr 8, 2013
#94 dd...@chromium.org
Bulk moving issues from M-27 to M-28.
Labels: -M-27 MovedFrom-27 M-28 bulkmove
Apr 8, 2013
#95 dd...@chromium.org
Bulk moving issues from M-27 to M-28.
May 8, 2013
#96 bugdro...@chromium.org
This issue is Pri-1 but has already been moved once, therefore lowering to Pri-2 and moving to next milesone.
Labels: -Pri-1 -M-28 MovedFrom-28 M-29 Pri-2
Jun 10, 2013
#97 nkostylev@chromium.org
(No comment was entered for this change.)
Labels: -M-29 M-30
Jul 11, 2013
#98 nkostylev@chromium.org
(No comment was entered for this change.)
Status: Started
Labels: Iteration-86
Jul 16, 2013
#99 bugdro...@chromium.org
------------------------------------------------------------------------
r211779 | nkostylev@chromium.org | 2013-07-16T09:20:18.735661Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/login/parallel_authenticator.h?r1=211779&r2=211778&pathrev=211779
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/login/login_performer.cc?r1=211779&r2=211778&pathrev=211779
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/chrome_browser_main_chromeos.cc?r1=211779&r2=211778&pathrev=211779
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/login/login_performer.h?r1=211779&r2=211778&pathrev=211779
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/login/authenticator.h?r1=211779&r2=211778&pathrev=211779
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/login/mock_authenticator.cc?r1=211779&r2=211778&pathrev=211779
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/login/mock_authenticator.h?r1=211779&r2=211778&pathrev=211779
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/login/parallel_authenticator_unittest.cc?r1=211779&r2=211778&pathrev=211779
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/login/parallel_authenticator.cc?r1=211779&r2=211778&pathrev=211779

[cros] Cleanup login auth stack: remove ClientLogin legacy code

* Remove Autheticator::RetryAuth() - was used to retry ClientLogin
* Remove Authenticator::AuthenticateToLogin() login_token, login_captcha parameters
* LoginPerformer - remove all code related to bringing up ScreenLock and dealing with ClientLogin results
* ParallelAuthenticator - make NEED_NEW_PW, HAVE_NEW_PW and ONLINE_FAILED states obsolete (ClientLogin related).
* ParallelAuthenticator unit_test - remove related test cases, always use OAuth code path

BUG=106538, 157246, 260219
NOTRY=true

Review URL: https://chromiumcodereview.appspot.com/18686006
------------------------------------------------------------------------
Jul 16, 2013
#100 nkostylev@chromium.org
Legacy code was removed. I'll monitor autotest buildbots if more cleanup will be needed. Work will continue in context of  issue 260219 .
Status: Fixed
Jul 25, 2013
#101 bhaves...@chromium.org
Dev fix, closing.
Status: Verified
Sign in to add a comment

Powered by Google Project Hosting