Skip to content
This repository has been archived by the owner on Nov 29, 2018. It is now read-only.

findElement hangs on a page with error log full of "Ignoring pending about:document-onload-blocker request" #4757

Closed
lukeis opened this issue Mar 4, 2016 · 48 comments

Comments

@lukeis
Copy link
Member

lukeis commented Mar 4, 2016

Originally reported on Google Code with ID 4757

Description: FF (v14, v15, v16) didn't time out and run infinitely.

Reproduce the issue: Selenium code in attachment.

Environment:
Selenium: 2.25.0
OS: Windows 7 and Linux

Have some Test-Reports:
FF14: https://saucelabs.com/tests/37e8613d9417485ab4edaae819de93ec
FF15: https://saucelabs.com/tests/cf9cb86e2d744a90a09074e9a96547ba
FF16 https://saucelabs.com/tests/9484057654fd4c9a8ef662de4d1b1984

Countercheck:
IE timed out as expected with 
org.openqa.selenium.TimeoutException: Timed out after 5 seconds waiting for presence
of element located by: By.xpath: //div[@class='tb-dropdown-body']//div[@class='no_items']

IE 9: https://saucelabs.com/tests/dc2f61554b9e4453a892cf35fdcbd7e8?action=fullscreen


History:
If you wonder about the guy on that webpage. It's actually not my issue - it's coming
from that thread: https://groups.google.com/forum/?fromgroups=#!topic/selenium-users/P-KM06gjeYY
and I managed to extract the problematic webpage. Don't ask me, why the WebApp was
providing that guy to me in that moment ...

Reported by michael.wowro@it-kosmopolit.de on 2012-11-09 18:02:12


- _Attachment: [PoojaTest.java](https://storage.googleapis.com/google-code-attachments/selenium/issue-4757/comment-0/PoojaTest.java)_
@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

at least the following functions show the same misbehaviour:

textToBePresentInElement
textToBePresentInElementValue
visibilityOfElementLocated

Reported by michael.wowro@it-kosmopolit.de on 2012-11-10 10:29:43

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Reported by barancev on 2012-11-13 08:52:05

  • Labels added: Lang-Java

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

It has nothing to the ExpectedConditions, simple findElement hands on this page.

A simplified reproduction scenario is (in groovy):

import org.openqa.selenium.*
import org.openqa.selenium.firefox.*

driver = new FirefoxDriver();
driver.get("http://it-kosmopolit.de/20121108153528/index.html");
Thread.sleep(2000)
println("checkpoint 1");
dropdown = driver.findElement(By.xpath("//div[@class='tb-dropdown-body']//div[@class='no_items']"))
println("checkpoint 2");

Reported by barancev on 2012-11-14 22:59:57

  • Labels added: Browser-Firefox
  • Labels removed: Status-Untriaged, Lang-Java

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I have encountered the same problem. Selenium 2.25.0 correctly throws a NoSuchElementException
under IE 9 and Chrome 23.0 when searching for a non-existent element by element ID.
Under FF17.0.1, it hangs indefinitely...I've waited a minute or two. I have a relatively
self-contained example (20 lines of code and a zip file containing the page) that I
can provide if it helps.

Reported by chris.l.merrill on 2012-12-17 19:30:56

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

@chris.l.merrill: FF17 support was added in Selenium 2.27, please upgrade.

Reported by barancev on 2012-12-17 19:53:06

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Ahhh....my apologies.  I looked at http://seleniumhq.org/download/ for the latest version
and found only 2.25.  But I see now that I should have been looking at https://code.google.com/p/selenium/downloads/list


Thanks - I've tested and it 2.28 works as expected with FF17 :>

Reported by chris.l.merrill on 2012-12-17 20:39:05

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I run into the same issue too.. I think correcting the first download link (http://seleniumhq.org/download/)
would be a time saver for others.

Reported by culudamar on 2012-12-20 09:45:35

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Hi, I recently came across this thread because my Firefox is hanging when I call FindElement
or FindElements. By "hanging," I mean, Firefox remains on the same page, and then returns
the Elements I was looking for almost 5 minutes later. I updated to FireFox 17 and
Selenium 2.28.0, but I'm still having the same problem. Does this sound like it would
be related to what was addressed for FindElement recently? 

Reported by danielle.song@makerstudios.com on 2013-01-04 07:11:52

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I'm using Mac OS X 10.7.5 and with multiple versions of Firefox as well as uninstalling
and reinstalling I am able to reproduce something like this bug using find_elements_by_name.
My repro script is available at https://gist.github.com/anastasia11/0bdaac342ab01d3a8891

Reported by Nastia.Rubanik@deliveryhero.com on 2013-02-06 13:54:10

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Not able to reproduce on Firefox 18.0.2 on OS X 10.7.4. 18.0.2 Just came out tonight
so we'll retest on 10.7.5 as well.

Reported by medwards@walledcity.ca on 2013-02-06 21:39:28

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I just tried this with FF 20 and Selenium 2.32.0 and I still encounter this issue. Any
pointers on workarounds till this issue is resolved?

Reported by pooja.mistry@smileymedia.com on 2013-04-12 06:13:36

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Any update on this?

Reported by madhurissanap on 2013-10-08 13:29:09

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I am observing the same issue.  Webdriver find_element calls are hanging with FF 28
and Webdriver 2.41.0.  In the Webdriver log, the "Ignoring pending about:document-onload-blocker
request" repeat indefinitely until I cancel the webdriver call on the command line.

I am observing the same behavior when I execute the tests on Linux running 'Mozilla
Firefox 24.4.0' with Webdriver 2.41.0.

I am using the 'unstable' firefox profile load strategy (Python: fp.set_preference("webdriver.load.strategy",
"unstable")).  Tracking down the webdriver log message led me to nsCommandProcessor.js
(https://code.google.com/p/selenium/source/browse/javascript/firefox-driver/js/nsCommandProcessor.js?spec=svn8c3c65772d57ac8493522613a6b50674864a771a&r=8c3c65772d57ac8493522613a6b50674864a771a).
 Oddly, a false should be returned in the DelayedCommand.prototype.shouldDelayExecutionForPendingRequest_
method, but for some reason the "if (this.loadGroup_.isPending())" conditional is still
met. 


Reported by aaronbriel on 2014-04-25 15:01:29

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

^^ aaronbr...@gmail.com, Were you able to fix it and how? I am seeing the exact same
issue on FF 28, 27, 24, and WebDriver 2.39.0, 2.41.0. I have tried everything!

Reported by ryanhnarakis on 2014-04-26 06:57:35

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

ryanhnar, I have not been able to fix this.  However, I did notice that the same issue
does NOT occur when I run my tests on another company site (I have a framework that
runs on various websites of products).   So, this is the same framework, same versions
of everything except the site being tested.  This site is using new versions of dojo
- and the main difference on the page source is that the one that hangs has the following
in it:
<script type="text/javascript"
            src="REDACTED/dojo-1.9.3/dojo.js"></script>
 <script type="text/javascript" src="REDACTED"></script>

Reported by aaronbriel on 2014-04-28 13:09:52

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Perhaps there is an issue with webdriver (or firefoxdriver) and the new version of dojo?

Reported by aaronbriel on 2014-04-28 13:10:56

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I've updated to Firefox 29 and the issue persists.  Any webdriver calls hang on a page
that has dojo-1.9.3 /dijit-1.9.3, and ends up failing with:

    try: self.driver.find_element_by_css_selector(what)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
line 367, in find_element_by_css_selector
    return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
line 655, in find_element
    {'using': by, 'value': value})['value']
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
line 164, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py",
line 347, in execute
    return self._request(command_info[0], url, body=data)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py",
line 415, in _request
    resp = opener.open(request)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/urllib2.py", line
400, in open
    response = self._open(req, data)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/urllib2.py", line
418, in _open
    '_open', req)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/urllib2.py", line
378, in _call_chain
    result = func(*args)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/urllib2.py", line
1207, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/urllib2.py", line
1180, in do_open
    r = h.getresponse(buffering=True)
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/httplib.py", line
1030, in getresponse
    response.begin()
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/httplib.py", line
407, in begin
    version, status, reason = self._read_status()
  File "/opt/system-builds/python-build/python-2.7/lib/python2.7/httplib.py", line
371, in _read_status
    raise BadStatusLine(line)
BadStatusLine: ''

Is there any progress on this issue?

Reported by aaronbriel on 2014-04-30 13:14:11

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

After our product upgraded to Dojo1.9.3(before is Dojo1.5), we met this issue too: hang
with log 'Ignoring pending about:document-onload-blocker request'. If we can't fix
this, does anyone find any workaround? Thanks

Reported by cccaseychang on 2014-07-22 07:37:49

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Still in selenium 2.43.1 and Firefox 31

Reported by grzegorz.halat on 2014-09-29 22:40:19

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Is there a new reproduction case?

The page mentioned in the comment no.3 has been changed and we have no reproduction
case now.

Reported by barancev on 2014-10-01 18:45:19

  • Status changed: NeedsClarification

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Still in Selenium 2.43.1 FF31 and FF32.0.3

Reported by vstrizhkova on 2014-10-02 09:44:36

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

"Still having the issue" is not a reproduction scenario :)

Reported by barancev on 2014-10-02 10:52:08

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

In my case it is problem with chat application which is using long pooling.
Changing nsCommandProcessor.js DelayedCommand.DEFAULT_SLEEP_DELAY to 1 solved problem.

Reported by grzegorz.halat on 2014-10-02 20:43:36

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

For me it is being triggered by elements on the dialogs

Reported by vstrizhkova on 2014-10-03 09:14:34

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016


I have the same issue, but it look hard to write a reproduction scenario, as it seems
to work with the simple cases. Still, here is what I noticed:
I have a page which keeps a long-polling connection opened to some server. And I have
a test in python, which does the following:

def test_xxx():
  self.retreive_from_db()
  self.open_the_page()
  self.look_for_some_element_on_page()

In this case the test hangs on the self.look_for_some_element_on_page() step. I have
lots of 'Ignoring pending about:document-onload-blocker request' in the webdriver log.

But when I tried the code like this (open the page first):

def test_xxx():
  self.open_the_page()
  self.retreive_from_db()
  self.look_for_some_element_on_page()

The test seem to work fine, nothing hangs. So maybe this help you, guys, to get the
problem fixed.

Ubuntu 14
selenium (2.43.0)
Firefox 32.0.3

Reported by kostadinov.zlatko on 2014-10-15 13:14:04

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Unfortunately we have no reproduction case for this issue anymore.
Feel free to request reopening if you have an executable reproduction scenario.

Reported by barancev on 2014-11-11 21:18:51

  • Status changed: Invalid

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

we ran into the same problem here with latest Firefox webdriver, by testing a realtime
web aplication which utilizes http long polling. The symptom is that automation hang(stopped)
after the web page loaded, by digging a bit into the webdriver source code and logs,
we found that the webdriver was getting into the "shouldDelayExecutionForPendingRequest"
loop infinitely, we tried to print out what the pending request is, and got huge amout
of these:
...
webdriver [141209 22:45:43.33] [156.482s] [fxdriver.DelayedCommand] Ignoring pending
about:document-onload-blocker request http://webchat.test.com/xx/xxServlet?format=json
webdriver [141209 22:45:43.44] [156.589s] [fxdriver.DelayedCommand] Ignoring pending
about:document-onload-blocker request http://webchat.test.com/xx/xxServlet?format=json
webdriver [141209 22:45:43.54] [156.691s] [fxdriver.DelayedCommand] Ignoring pending
about:document-onload-blocker request http://webchat.test.com/xx/xxServlet?format=json
...

The "http://webchat.test.com/xx/xxServlet?format=json" is exactly the long polling
request which is used in this web app, and we found the following conditional statement
in nsCommandProcessor.js caused this:

if (numPending && !hasOnLoadBlocker) {
      fxdriver.logging.info('Ignoring pending about:document-onload-blocker ' +
        'request');
      // If we only have one pending request and it is not a
      // document-onload-blocker, we need to wait.  We do not wait for
      // document-onload-blocker requests since these are created when
      // one of document.[open|write|writeln] is called. If document.close is
      // never called, the document-onload-blocker request will not be
      // completed.

      return true;
    }

As in http long polling application, there is always a living http request to server
even after the web page loads completely, in our case, the http request sent to server
one after another every 30 seconds, if no update data returned from server, the http
request held for 30 seconds then ended with 200 OK but a blank response. Can we bypass
this kind of long polling request by adding extra logic to check some other properties
for example state during delay period?

Reported by gmdavy on 2014-12-10 09:36:51

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Just found this issue was closed as invalid, could we reopen it as I believe this is
not fixed?

Reported by gmdavy on 2014-12-10 09:44:00

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

The issue was closed because there is no executable reproduction scenario.

Reported by barancev on 2014-12-10 09:59:24

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Is there possibility to provide a fix given cause analysis above for the long polling
scenario is right? 

Reported by gmdavy on 2014-12-11 02:46:11

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Blind fixing is not the best idea. Is there a demo-site in the world that can be used
as a sample of "long polling" page causing the issue? What kind of "other properties"
do you suggest to check? How can we decide this without a sample?

Reported by barancev on 2014-12-11 05:17:21

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Ok, thanks, understood that. I got a test sample file(java) which can recreate this
issue somewhat against a public site. however I just can not attach it here because
it shows the attachment storage quota exceeded, please let me know how I can send you.

Reported by gmdavy on 2014-12-12 11:50:00

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

You can use https://gist.github.com/ to share the sample

Reported by barancev on 2014-12-12 14:43:40

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I copied the sample code here: https://gist.github.com/anonymous/ec32065e7e002724865f
With this sample, I saw at least 2~3 minutes hang in the last two actions while in
our real automation, it hangs forever.

Reported by gmdavy on 2014-12-15 09:30:59

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

@barancev, are you able to reproduce this issue with the sample? if yes we'll need to
reopen this ticket to track the status for fixing it

Reported by gmdavy on 2015-01-12 02:31:51

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

any update for this?

Reported by gmdavy on 2015-01-20 08:34:33

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

This issue is still presente in selenium 2.45.0 and firefox 36.0
hangs during  
private native int socketRead0(FileDescriptor fd,
                                   byte b[], int off, int len,
                                   int timeout)
        throws IOException;

ignoring timeout

Reported by andre.lancia on 2015-03-05 07:24:16

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I had a similar problem and setting up the driver like this fixed it for me:

DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability(CapabilityType.PAGE_LOADING_STRATEGY, "none");
WebDriver result = new FirefoxDriver(caps);

Reported by mail@eswapnil.com on 2015-04-14 20:21:50

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I had: "Ignoring pending about:document-onload-blocker request" and found out that it
was caused by signalR long polling requests. Selenium has been waiting for long polling
call to end and got timeout. Guys check out if you don't have any long polling requests
It may be a clue.

Reported by Powalec666 on 2015-04-29 08:33:05

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Hi All,

I use selenium webdriver 2.47 and Firefox version is 40. I have similar issue, throwing
error "Ignoring pending about:document-onload-blocker request" after login. It is because
of long polling. I confirmed by hacking the response of long polling as 500 using fiddler,
then it worked fine.

My webpage defer loads the dom elements so using desire capabilities as mentioned above
in the thread didn't work for me. As soon as my webpage opens selenium starts find
the dom element, not waiting for the dom to load. 

please help.

Thanks,
Vijay.

Reported by nk.viju on 2015-09-09 20:14:20

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

To solve the problem I rebuild selenium using following guide: http://shengwangi.blogspot.com/2014/08/how-to-build-selenium-from-source.html
This generated my custom version of jar file.
Based on comment #28 I updated nsCommandProcessor.js and just added:
(request.name.indexOf('<your comet request>') > -1)
to
hasOnLoadBlocker = hasOnLoadBlocker ||
                         (request.name == 'about:document-onload-blocker');
so it would ignore the comet call.
nsCommandProcessor.js is under javascript\firefox-driver\js folder

good luck

Reported by ultrame on 2015-09-09 21:05:49

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I use selenium in .Net, is there any similar option in C#?

Thanks!

Reported by nk.viju on 2015-09-10 17:41:53

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Meant to run it in local machine not using grid.

Reported by nk.viju on 2015-09-10 19:32:02

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

hasOnLoadBlocker = hasOnLoadBlocker ||
                         (request.name == 'about:document-onload-blocker') || (request.name.indexOf('about:document-onload-blocker')
> -1);

this is how I tried. Is that correct?

Reported by nk.viju on 2015-09-14 19:11:47

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

'about:document-onload-blocker' is already handled, so <your comet request> is not that.
You need to open firebug and see which request keeps repeating or doesn't get closed.
The request might have 'comet' in the name. Just need to add any part of that request
URL, hopefully it is unique enough so that selenium doesn't skip waiting for other
similar requests.

Reported by ultrame on 2015-09-15 00:29:52

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

ok thanks. I'll try and let you know.

Reported by nk.viju on 2015-09-15 03:24:36

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Great!!!

It worked.

Thanks a lot.

Reported by nk.viju on 2015-09-15 17:19:16

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Reported by luke.semerau on 2015-09-17 18:16:19

  • Labels added: Restrict-AddIssueComment-Commit

@lukeis lukeis closed this as completed Mar 4, 2016
@SeleniumHQ SeleniumHQ locked and limited conversation to collaborators Mar 4, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant