Skip to content
This repository was archived by the owner on Nov 29, 2018. It is now read-only.

Firefox Driver 2.45 is 100% slower than 2.44 #8551

Closed
lukeis opened this issue Mar 4, 2016 · 90 comments
Closed

Firefox Driver 2.45 is 100% slower than 2.44 #8551

lukeis opened this issue Mar 4, 2016 · 90 comments

Comments

@lukeis
Copy link
Member

lukeis commented Mar 4, 2016

Originally reported on Google Code with ID 8551

Hello, 
I updated today from 2.44 to 2.45 (selenium-server-standalone-2.45.0.jar). ran my tests
on firefox (35) . The running time has grown by 100%. When I rolled back to 2.44 it
fixed the problem. My tests run mainly inside iFrames , but not only.
Thank you.



Selenium version:2.45
OS:Windows7
Browser:Firefox 
Browser version: 35.0.1




Reported by pinchasn81 on 2015-03-01 17:42:04

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Reported by jmleyba on 2015-03-01 21:20:57

  • Labels added: Browser-Firefox, Component-WebDriver
  • Labels removed: Status-Untriaged

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Yes, it is a known issue caused by revision e581237f8288

Reported by barancev on 2015-03-02 11:55:44

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Hello ,
This is a major problem for us.Is there a possibility for a quick fix?
Thank you.

Reported by pinchasn81 on 2015-03-02 12:14:12

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Patches are welcome :)

Reported by barancev on 2015-03-02 12:49:54

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Hi, the performance degradation is also on Windows - making it impossible to work with.
Please fix... 

Its priority is defiantly not medium  - should be High.

Thanks!

Reported by ilievsky on 2015-03-02 13:20:09

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

We do watch build time on the Selenium CI server, and we know for sure that not all
environments are subject to this issue. See e.g. [1], [2], [3] -- there is no sign
of performance degradation.

Yes, there is a problem. And we appeal to the community -- if the priority is really
high please spend more time to help us identify exact conditions where it appears,
and may be even produce a patch to fix it. You're welcome to the open source world
:)

[1] http://ci.seleniumhq.org:8080/job/Firefox%2031%20Linux%20Synthesized%20Java%20Tests/buildTimeTrend
[2] http://ci.seleniumhq.org:8080/job/Firefox%2031%20Linux%20Native%20Java%20Tests/buildTimeTrend
[3] http://ci.seleniumhq.org:8080/job/Firefox%2035%20Linux%20Synthesized%20Java%20Tests/buildTimeTrend

Reported by barancev on 2015-03-02 14:29:07

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Yes, we're seeing this in both Windows and Linux environments using Firefox.

Reported by titusfortner on 2015-03-02 17:50:01

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Seeing the same thing here on 64-bit Windows 7 (build time tripled) and 64-bit Ubuntu
12.04 Server (headless) (build time quadrupled.

Reported by terry.lacy on 2015-03-03 21:38:22

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Sorry, I neglected to mention my FF and Selenium versions before the increase in build
time.  I was on FF34/Selenium 2.44.0 and upgraded to FF36/Selenium 2.45.0.
I don't see any particular pattern for what's taking longer.  It looks like my tests
(over 300 of them) pretty much take longer across the board.  I do use quite a bit
of javascript to query the dom during my tests.  Dunno if that affects the dramatic
increase in build time.  Note that I saw much more dramatic increases in build time
(300-400% increase as opposed to 100%).

Reported by terry.lacy on 2015-03-03 21:45:01

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Hi,
Same issue coming up for us. We have a test suite of around 350 tests which used to
run in 2 hours, now it's running for over 5 and it's only 3/4 done. Tests run on a
64-bit Ubuntu Server (headless), recently upgraded from FF34/Selenium 2.44 to FF35/Selenium
2.45.

Agreed, priority should be High.

Thanks

Reported by maciej.czech.1989 on 2015-03-05 21:53:21

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I am also seeing this issue on FF36/Selenium 2.45.  I downgraded to FF32/Selenium 2.44
and things are working as normal.

Reported by neelavr on 2015-03-05 21:56:08

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

same issue for me: Linux + FF36/Selenium 2.45

Reported by thomas@nuroa.com on 2015-03-06 16:20:24

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Same deal for me. Execution time has roughly doubled on every single test we have immediately
on update.

The ticket claims to only affect Linux, but I think that should be updated, as we're
on Macs & Win8.1 and are seeing the same issue on both. As with the others commenting
above, downgrading to our previous setup seems to have resolved it for us just as quickly
as the problems arrived

Currently: 
- OSX10.10.2 / FF 36 / Selenium 2.45
- Win8.1 / FF34 / Selenium 2.44

Before: 
- OSX10.10.2  / FF 36 / Selenium 2.45
- Win8.1 / FF34 / Selenium 2.44

Reported by steve@epactnetwork.com on 2015-03-06 16:38:09

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I agree with #17.  This appears to severely impact performance across all platforms:
Windows 7, Mac, Linux ...?  Also, just to be especially concise, the title should indicate
that the run time is 2-3 times as long with the new version, not just double (I'm apparently
an outlier with my Ubuntu tests taking 4 times as long).

Reported by terry.lacy on 2015-03-06 16:44:32

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Oh god sorry, let me update my comment #17, I've jumbled my platform list :P it's too
early in the morning

Currently: 
- OSX10.10.2 / FF 36 / Selenium 2.45
- Win8.1 / FF36 / Selenium 2.45

Before: 
- OSX10.10.2  / FF 34 / Selenium 2.44
- Win8.1 / FF34 / Selenium 2.44

Reported by steve@epactnetwork.com on 2015-03-06 16:46:38

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

in my case performance goes down from +-20sec to 60-180sec - last ESR FF, Windows,
noticable increase on getAtribute("class") - every single one of them now take up to
0.5 sec

Reported by memnon666 on 2015-03-08 22:01:03

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Issue 8585 has been merged into this issue.

Reported by barancev on 2015-03-09 10:36:12

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Upgraded from firefox 34 and selenium 2.44 to firefox 36 and selenium 2.45. All on osx
10.10.3. All of my tests take about 3.5 times longer consistently. Wondering if there
is a fix being developed or if this issue was able to be recreated by a developer.
Also any news on how long a fix might take to arrive? 

Reported by benjamin.orgera on 2015-03-10 00:06:18

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Similar issue for me although I'm using FF ESR 31.2, Java 1.7v45, and WIn 7 - these
items have been in place and stable for quite some time

When I updated from 2.44 to 2.45 (selenium-server-standalone-2.45.0.jar), all my locator
queries (I use xpath exclusivly) executed very slow. No iframes involved.
Rolling back to 2.44 resolved the problem

Thank you.

Selenium version:2.45
OS:Windows7
Browser:Firefox  ESR 31.2

Reported by william.s.charlton on 2015-03-11 19:10:11

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

The Watir team (mostly Alex) has proved that it was a garbage collection issue in httpds.js
file.

This is the fix: https://github.com/SeleniumHQ/selenium/commit/d6983f302607ec895ae150078f094ba179fd6204

There should be a point release for this issue soon.

Reported by titusfortner on 2015-03-12 16:00:53

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Just checking to see when the release will be cut for this fix?

Thanks!

Reported by ben@pithoslabs.com on 2015-03-15 20:01:32

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

A +1 for a bug fix release in case the new release 2.46 is going to be a while away

Reported by smita8504 on 2015-05-11 20:13:53

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

#70 - Watirspec tests with the gem you reference show no substantial difference between
Firefox & Chrome - https://travis-ci.org/watir/watir-webdriver/builds/54052030 vs same
Watir code without the fix: https://travis-ci.org/watir/watir-webdriver/builds/54017021
This is the right fix, and without looking at your setup, I can't tell you why you
aren't seeing the same results we did.

Reported by titusfortner on 2015-05-11 21:06:17

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Please get this fixed asap.

Reported by kunal.sharma304 on 2015-05-12 07:05:22

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Does no-one else think this is farcical?

Reported by craig.miles0712 on 2015-05-12 07:20:22

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Hi we have been troubled by this issue lately. 

Our workaround for Ruby on Rails
(1) Use firefox v34 and downgrade selenium-webdriver to v2.44

OR (2)
a. Git clone my fork'd repo (as shown below) which is based on 2.45 release with the
fix (thanks to Titus for the hint)
https://github.com/Katsukisan/selenium

b. execute ./go //rb:gem:release
c. copy the gem to ./vendor/gems/ (or any folder which you prefer)
d. gem unpack <gem>, this will create ./vendor/gems/selenium-webdriver
e. in the Gemfile, gem 'selenium-webdriver', '2.45.0.dev3', path: './vendor/gems/selenium-webdriver'

Reported by gylsvc on 2015-05-12 10:06:22

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Titus,  I agree my results seem strange.  No idea what is causing it, initial attempts
to find a smoking gun failed, it seems like everything webdriver related is slower,
I've dug down a bit hoping to find a single thing, that is slow, and unless it is locating
and parsing dom stuff, I can't find a culprit.

I'm at the point where I suspect the overall slowness is being caused by something
else.  And that the initial improvement I saw with the 'fixed' version represents what
fixes things entirely for other folk

I'll chase my overall FF perf issue in Watir General google-group after I make a script
others can easily run, see if it is the site I'm testing, or something specific to
my system (Macbook pro, OSX 10.10.3)

Meanwhile I'm in favor of releasing the 'fix' we have so far as it did make a difference
for me, and others who have tested it.  

Reported by SQAPro on 2015-05-13 17:29:25

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I am also facing same issue with my automation run. Automation run time is increased
by 100 %. I am using FF30.0/2.45 on windows 8. Any workaround for this issue?

Reported by shishirdwivedi2008 on 2015-05-18 13:15:06

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

> Any workaround for this issue?

Build from source

Reported by vlad.filippov on 2015-05-20 14:05:21

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

This is an absolute joke. My team and I are now considering the ramifications of scrapping
our entire suite of 15k test and finding an alternative. This issue has been present
for MONTHS, has negatively impacted the quality of our product, and has not been meaningfully
addressed.

Reported by KennyJohnsonIII on 2015-05-20 19:38:04

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

refer to this thread for updates. The selenium team seems to at least pay attention
to it. https://github.com/SeleniumHQ/selenium/issues/383

Reported by KennyJohnsonIII on 2015-05-20 19:57:30

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Issue 383 was closed. In hopes of getting more attention to this issue opened a new
issue: https://github.com/SeleniumHQ/selenium/issues/559

Reported by vlad.filippov on 2015-05-21 19:00:00

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

we are no longer paying attention to google code issues... all issue activity should
take place on github.

Also as mentioned earlier this was fixed and is just awaiting a release or you can
build it yourself or download the latest for selenium's ci:
http://ci.seleniumhq.org/selenium-server-b4692eaa05633fc96c69b7198ed2040951c8a1c6.jar

Reported by luke.semerau on 2015-05-21 19:10:45

  • Status changed: Fixed

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

For all the people who have this issue starred but have not yet noticed, Selenium 2.46.0
has been released. http://docs.seleniumhq.org/download/

Reported by Sophosis on 2015-06-04 21:39:56

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Why doesn't 2.46 java have a maven repo?

Reported by team@supremesharkbot.com on 2015-06-05 01:05:07

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Looks good fellas. 

I just went to the pom.xml of my project, threw in 2.46 as my selenium version, updated
my machine's Firefox to 38, and initial tests are giving me pretty much the same speeds
on 2.46/FF38 as I was getting with 2.44/FF34.

Note: we have tests running on OSX, ArchLinux, and Win8.1, and all 3 were affected
by this issue, so I'll have to check the rest tomorrow, but for now I've only confirmed
this on my Mac

In case it helps anybody, on updating from 2.44 to 2.46 I was getting an issue with
a NoClassDefFoundError with HttpClient trying to run tests that were grooving along
nicely in 2.44. In case one of you guys also updating your pom has the same problem,
just throw the following dependency in your pom.xml and you'll be good to go:

<dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.3.4</version>
 </dependency>

Reported by steve@epactnetwork.com on 2015-06-05 02:21:35

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I upgraded jars to 2.46, it also takes same time as 2.45. Seems there is something wrong
with 2.45 & 2.46. 2.44 used to take 3hrs to run 600 test and now with 2.45 and 2.46
it takes around 6.5 hrs.

Reported by mailforpriyanshu on 2015-06-10 06:34:46

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Selenium 2.46 with Firefox 38 seems to be "back to normal" for me, as for
other users.

Thanks to the team for finally getting this resolved.

Reported by craig.miles0712 on 2015-06-10 13:05:51

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I upgraded to 2.46 and Firefox 38 but don't see any change when compared to the 2.45/Firefox
37 combination. I am thinking of going back to 2.44 (and an older version of Firefox)
and check how things fare.

Reported by A.M.Bharath on 2015-06-10 16:30:38

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Upgraded to selenium 2.46 and Firefox 38 and the performance is similar to Selenium
2.44/Firefox 33! No more double and triple the test time! Why this wasn't done as a
point release is still a mystery.

Thanks selenium team

Reported by maciej.czech.1989 on 2015-06-10 16:39:23

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Did anybody experience that the good old "Command failed to close cleanly. Destroying
forcefully (v2)" issue is back with the new Firefox Driver? 

Reported by patakymr on 2015-06-10 16:49:35

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

yeah, actually I have noticed that in windows. It seems intermittent, and
it actually doesn't seem to have any adverse affect, i'm actually glad the
binary at least "fails" to a code path that does this instead of just
hanging. Maybe make a new thread on Github though if you want it addressed.
It seems pretty clear from the way this issue played out that the
developers on this project aren't paying attention to google groups.

Reported by controll.face on 2015-06-19 05:26:34

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

I updated to Selenium 2.46 & FF 38.0.1 and it is much slower than before.

Old setup: Selenium 2.44 & FF 33.1.1 -> around 90 minutes to finish.
New setup: Selenium 2.46 & FF 38.0.1 -> around 150 minutes to finish.

I am using linux 14.04 (64-bits).
I hope you realise that this must be fixed.
Thank you for your effort.

Reported by lorenzo.fidalgo@openbravo.com on 2015-06-29 07:32:14

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Thanks for fix!

Reported by ryan.p.chap on 2015-07-04 19:22:23

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Just an update regarding the additional slowness I was having with firefox (reported
back in may)

That specific issue appears to be related to Avast's anti-virus product for the mac.
 even with their firefox extension turned off, FF was terribly slow with webdriver.
 I had to entirely remove avast, and once I did that, BINGO things were back to their
normal speedy performance

Reported by SQAPro on 2015-07-21 21:46:57

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

@ #94: I have the same issue with Avast slowing down Selenium. Everytime Selenium is
actively working, Avast goes up to 90% of CPU.

I disabled Avast in Settings > Disable FileShield and the problem went away. It seems
to be okay to have WebShield active.

Reported by niklas.nordstrom2 on 2015-08-28 11:45:05

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Y'all are replying to a bug report that nobody's looking at any more.  Google code is
going away.  Selenium has moved to github (https://github.com/SeleniumHQ).
Bug reports should go here now: https://github.com/SeleniumHQ/selenium/issues

Reported by terry.lacy on 2015-08-28 15:05:40

@lukeis
Copy link
Member Author

lukeis commented Mar 4, 2016

Reported by luke.semerau on 2015-09-17 18:25:55

  • Labels added: Restrict-AddIssueComment-Commit

@lukeis lukeis closed this as completed Mar 4, 2016
@SeleniumHQ SeleniumHQ locked and limited conversation to collaborators Mar 4, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.