Navigation Menu

Skip to content
This repository has been archived by the owner on Nov 29, 2018. It is now read-only.

[SafariDriver] implement the interactions API #4136

Open
lukeis opened this issue Mar 3, 2016 · 50 comments
Open

[SafariDriver] implement the interactions API #4136

lukeis opened this issue Mar 3, 2016 · 50 comments

Comments

@lukeis
Copy link
Member

lukeis commented Mar 3, 2016

Originally reported on Google Code with ID 4136

mouse functions are not working for safari.

example

Mouse actions = new Actions (driver);
mouse.moveToElement (series). build (). perform ();
mouse.moveToElement (generos.get (0)). build (). perform ();

it does not work for safari

Reported by flpsan on 2012-06-22 16:59:08

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

The interactions API hasn't been implemented yet.

Reported by jmleyba on 2012-06-23 16:31:11

  • Status changed: Accepted
  • Labels added: Component-WebDriver, Browser-Safari
  • Labels removed: Status-Untriaged

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Issue 4152 has been merged into this issue.

Reported by jmleyba on 2012-06-27 22:23:12

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

This issue was updated by revision r17393.

Interaction API tests should be re-enabled once implemented for the SafariDriver.

Reported by jmleyba on 2012-07-10 03:09:28

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

I was just wondering, going through issue 2067, I ported the code sample in comment
#60 to Java and my colleague tested it out and it appears to work for mouse over in
Safari. I did the port in hopes to fix issue with FF and IE, but seems it works for
Safari too.

So then, from that, I assume as a temporary workaround, some of the interactions API
could be simulated with javascript code calls, at least for mouse and keyboard events?

Reported by mangaroo on 2012-07-11 21:05:27

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

I am also getting the same issue with safari browser.

Error :

Unknown command: mouseMoveTo (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 9 milliseconds
Build info: version: '2.25.0', revision: '17482', time: '2012-07-18 21:08:56'
System info: os.name: 'Windows 7', os.arch: 'x86', os.version: '6.1', java.version:
'1.6.0_30'
Driver info: driver.version: ReusableInternetExplorerDriver

Thanks,
Siva

Reported by sivaraaj on 2012-08-29 12:02:36

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

WorkAround : 

public boolean onMouseOver(WebElement element)
    {
        boolean result = false;
        try
        {
            String mouseOverScript = "if(document.createEvent){var evObj = document.createEvent('MouseEvents');evObj.initEvent('mouseover',
true, false); arguments[0].dispatchEvent(evObj);} else if(document.createEventObject)
{ arguments[0].fireEvent('onmouseover');}";
            JavascriptExecutor js = (JavascriptExecutor) driver;
            js.executeScript(mouseOverScript, element);
            result = true;
        }
        catch (Exception e)
        {
            e.printStackTrace();
            result = false;
        }
        return result;
    }

Thanks,
Siva

Reported by sivaraaj on 2012-08-29 13:27:50

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Can you provide the equivalent code for this method, please?

public static void dragTo(WebElement from, WebElement to)
    {
        Actions builder = new Actions(driver);
        Action dragTo = builder.clickAndHold(from).moveToElement(to).build();
        dragTo.perform();
    }

When will this issue be released? Thanks a lot!

Reported by carolina.glatstein on 2012-11-23 10:16:12

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

I am facing same issue on Safari for DragDrop element.

Any update on official Safari web driver? When we can get fix on above issue.  

Reported by mrinalheritageit on 2012-12-21 12:05:28

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Just found this today, haven't tried it out, but looks like good option for drag & drop
for Safari as well as a fallback for other browsers should the native drag & drop fail
for some reason:

Javascript based drag & drop for WebDriver...
http://ynot408.wordpress.com/2011/09/22/drag-and-drop-using-selenium-webdriver/

Reported by mangaroo on 2013-01-24 19:17:51

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Issue 5046 has been merged into this issue.

Reported by jmleyba on 2013-01-25 05:23:38

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Issue 5093 has been merged into this issue.

Reported by barancev on 2013-01-29 22:00:18

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Any update? Alternatively, is there a good place to start looking into how to implement
this feature?

i've got a moderately complex set of web tests that make extensive use of drag-and-drop
to test, well, drag-and-drop behaviour of an app. These tests work fine in Chrome and
FireFox - I want to make them work in Safari as well.

Reported by robertdw@twasink.net on 2013-04-16 12:26:20

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

FYI, if we're talking drag & drop support, the solution I mention in comment #10 works
in Safari. I tested it. 

So you'd just need to write a special wrapper method to drag & drop so that you use
the interactions API on all browsers except Safari for which you'd use javascript simulation.

Granted it's not realistic testing but it works as a workaround until this issue is
resolved.

Reported by mangaroo on 2013-04-16 20:58:43

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Issue 5809 has been merged into this issue.

Reported by jmleyba on 2013-06-20 17:44:31

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Anything new on this besides the JavaScript workaround?

Reported by thomas.kieffer@codecentric.de on 2013-06-27 11:54:46

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Is there any news/updates on this issue? I am having issues with this and Safari and
am wondering if the only current workaround is to use javascript.

Reported by jess.lawrence@axomic.com on 2013-08-12 10:02:43

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Same here. Presumably the JS workaround is the way to go. What I don't get is why it
is actually included in the product. It's by no means usable. 

Reported by java.artisan on 2013-08-12 11:05:35

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

On comment #18,

I haven't tried invoking any of the interactions API since I've come across this issue,
I just use the JS workarounds.

So what does happen when you invoke the unimplemented APIs? If it's just that nothing
happens, you have a valid point. If a some exception is thrown, that's at least ok
to me. 

I would figure some "unimplemented API" exception would be ideal to let the users know
this particular API is not yet available for use for the given driver. But all the
drivers should support (as in respond to) all the WebDriver APIs because they are driver
interfaces to it, and simply respond with an exception to indicate "I don't support
that right now" when an unsupported API is invoked, since we want a standardized way
to use the drivers.

Reported by mangaroo on 2013-08-12 17:53:52

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

By the way, for context, this topic seems quite important to some people. What's so
bad about javascript workaround in the meantime?

Is it because you "really must" test something with native mouse interaction, etc.?

Or you have trouble finding the right javascript workaround code to use?

Reported by mangaroo on 2013-08-12 17:57:15

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

It does raise an exception: "WebDriverException: Message: u"Unknown command: mouseMoveTo"

It's not that I don't want to use the javascript implementation, It's more that I wanted
to know if there are any limitations to using it (using the API's click() on a WebElement
does not always trigger javascript events and so using javascript to emulate it is
a must in some instances). 


Reported by jess.lawrence@axomic.com on 2013-08-13 08:33:25

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

From what I gather, there shouldn't be any limitation using the javascript route. What
I've heard (in terms of IE and maybe Firefox) though is that it may not be as realistic
in simulating actual mouse behavior (based on the internal implementation of the browser
driver) for the browser being automated. But for Safari, there's currently no difference,
you either have the functionality (through javascript) or not (without it).

Reported by mangaroo on 2013-08-13 16:05:07

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Just to clarify, the SafariDriver uses 100% synthetic events.  When the interactions
API is implemented, it will be done using synthetic javascript events from Selenium's
browser automation atoms library.

Reported by jmleyba on 2013-08-13 19:41:39

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Hi Selenium Team,

Is this issue fixed and what is the latest status of this issue..

Thanks,
Srinivas

Reported by vuppala.srinivas on 2014-03-05 09:45:41

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Issue 7037 has been merged into this issue.

Reported by barancev on 2014-03-05 21:00:32

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Please update us on this issue, thanks in advance.

God bless!

Reported by cbier90 on 2014-03-07 22:57:52

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Reported by jmleyba on 2014-03-10 01:05:34

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Safari is not a priority, sorry. But your patches are welcome!

Reported by barancev on 2014-03-10 06:35:12

  • Labels added: GettingInvolved

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

To anyone still interested this is the workaround method I used for moveToElement -->

Tested by me and works..

public void moveToElement(WebDriver driver, WebElement element, By locator) {
        if (!(driver instanceof SafariDriver)) {
            Actions builder = new Actions(driver);
            builder.moveToElement(element).perform();
        }
        else {
            JavascriptExecutor js = (JavascriptExecutor) driver;
            String locatorType = locator.toString().substring(3);
            String elem = "var elem = document;";
            if (locatorType.startsWith("id")) {
                elem = "var elem = document.getElementById(\""+locatorType.substring(4)+"\");";
            }
            else if (locatorType.startsWith("xpath")) {
                String snippet = "document.getElementByXPath = function(sValue) { var a = this.evaluate(sValue,
this, null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null); if (a.snapshotLength > 0)
{ return a.snapshotItem(0); } }; ";
                js.executeScript(snippet);
                elem = "var elem = document.getElementByXPath(\""+locatorType.substring(7)+"\");";
            }
            else if (locatorType.startsWith("className")) {
                elem = "var elem = document.getElementsByClassName(\""+locatorType.substring(14)+"\")[0];";
            }
            String mouseOverScript = elem + " if(document.createEvent){var evObj = document.createEvent('MouseEvents');evObj.initEvent('mouseover',
true, false);" +
                    " elem.dispatchEvent(evObj);} else if(document.createEventObject) { elem.fireEvent('onmouseover');}";
            js.executeScript(mouseOverScript);
        }
    }

Reported by nirmithadasun on 2014-08-01 08:16:50

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

For #38, what code exactly is giving you issue?

Reported by mangaroo on 2014-09-23 17:36:17

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Hi,

In regards to my problem :

Exception in thread "Thread-2" org.openqa.selenium.WebDriverException: Unable to uninstall
extension
Caused by: java.io.FileNotFoundException: /var/folders/9k/22zq2y2x1mj_pglnzq_g6rfr0000gp/T/SafariBackups4126064663359463498webdriver/Extensions.plist
(No such file or directory)

After looking a bit closely, its not the code in #38 that is at fault, but in:

            Actions builder = new Actions(driver);
        builder.clickAndHold(l_draggable);
        Thread.sleep(500);
        moveToElement(driver, By.id("droppable")); //Using the code in #38
        builder.release().perform();

Removing builder.release().perform(); solves my problem. (Since my problem has no beareance
on this thread I have removed it)...

Unfortunately I was unable to get the code in #38 to work regardless of my problem.
However the following worked for me:

    public void dragdrop(WebDriver driver, By ByFrom, By ByTo) {
        WebElement LocatorFrom = driver.findElement(ByFrom);
        WebElement LocatorTo = driver.findElement(ByTo);
        String xto=Integer.toString(LocatorTo.getLocation().x);
        String yto=Integer.toString(LocatorTo.getLocation().y);
        ((JavascriptExecutor)driver).executeScript("function simulate(f,c,d,e){var b,a=null;for(b
in eventMatchers)if(eventMatchers[b].test(c)){a=b;break}if(!a)return!1;document.createEvent?(b=document.createEvent(a),a==\"HTMLEvents\"?b.initEvent(c,!0,!0):b.initMouseEvent(c,!0,!0,document.defaultView,0,d,e,d,e,!1,!1,!1,!1,0,null),f.dispatchEvent(b)):(a=document.createEventObject(),a.detail=0,a.screenX=d,a.screenY=e,a.clientX=d,a.clientY=e,a.ctrlKey=!1,a.altKey=!1,a.shiftKey=!1,a.metaKey=!1,a.button=1,f.fireEvent(\"on\"+c,a));return!0}
var eventMatchers={HTMLEvents:/^(?:load|unload|abort|error|select|change|submit|reset|focus|blur|resize|scroll)$/,MouseEvents:/^(?:click|dblclick|mouse(?:down|up|over|move|out))$/};
" +
        "simulate(arguments[0],\"mousedown\",0,0); simulate(arguments[0],\"mousemove\",arguments[1],arguments[2]);
simulate(arguments[0],\"mouseup\",arguments[1],arguments[2]); ",
        LocatorFrom,xto,yto);
    }

//This is based on findings in : http://ynot408.wordpress.com/2011/09/22/drag-and-drop-using-selenium-webdriver/

Reported by Development@Gandomi.com on 2014-09-24 09:54:08

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

I am having the following piece of code in my script

Actions action = new Actions(driver);
action.sendKeys(month);
action.perform();

Since the reference is to a method(sendKeys) in Action.class, this execution fails
on safari.
SendKeys in action class is as follows:

public Actions sendKeys(WebElement element, CharSequence... keysToSend) {
    action.addAction(new SendKeysAction(keyboard, mouse, (Locatable) element, keysToSend));
    return this;
 }

Can you let me know what work around can be used.

Appreciate your response.

Thanks

Reported by bharathdesais on 2014-10-01 04:18:42

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

On comment 41, why would you do it this way? What's the requirement?

Because the common approach works:

WebElement.sendKeys("keys");

that would be my workaround.

Reported by mangaroo on 2014-10-01 04:39:59

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

On Comment 42, We are trying to build script for a web application to be tested on all
browsers and devices. 
When executed on devices following were few issues we faced:
1.When the tests are executed on devices, the native control of the device pops-up
for a drop down etc. 
2. When navigated from one page to another (pages having a scroll bar), the page control
remained at the bottom of the page instead of the top of the page.Hence we have to
do a pageup.

For both the issues faced following were the solutions we are using
1. We are sending keys to the dropdown in few cases, However we could use Select which
is specific to dropdown.
2.For pageup, the only solution we have is to use Actions
Actions action = new Actions(driver);
action.sendKeys(Keys.HOME);
action.perform();

Hence we want to avoid usage of Action in our framework so that its compatible with
all browsers and devices. 

This was one main reason for which I was looking out for a workaround.

Appreciate your response.

Reported by bharathdesais on 2014-10-02 00:42:11

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Issue 8447 has been merged into this issue.

Reported by jmleyba on 2015-02-02 16:53:35

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

moveToElement() is not working in any Safari version, not even 8.

The only option is writing a JavaScript function that will hover over the element.
BUT that doesn't cover all situations.
In cases where the mouse hover was implemented in the page using JavaScript, it will
work, but in the cases it was built using CSS pseudo elements, it won't work.
I tried a lot to make this work, and this answer also says it's not a "W3C trusted
event":
http://stackoverflow.com/a/17226753/705147

Reported by cbaldan@daitangroup.com on 2015-04-07 16:06:04

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

FYI, you know, when all else fails, while waiting for this feature (or bug fix), the
last resort option can be to use external tools to integrate with Selenium/SafariDriver
to handle the limitations (e.g. Java Robot class, AutoIt, Sikuli). Not clean/pretty,
but at least you're not stuck.

And if you need to figure out how to deploy this integration for Grid, something like
this may work:

https://gist.github.com/daluu/3b4746f6b672c49f7e8f

Reported by mangaroo on 2015-04-07 17:52:00

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Manga,

Thanks for the info. I looked up all the code from "PartialSeleniumGridIntegrationWithAutoItExample.java",
but I don't think any of those options will work with BrowserStack nor Saucelabs.

I also looked up your projects, but if I'm not mistaken, all options require some lib
or jar to be deployed and executed in the node machine. Given that my current client
uses BrowserStack, I had to drop support for Safari in a bunch of tests.

Reported by cbaldan@daitangroup.com on 2015-04-07 18:02:43

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

When testing javascript heavy sites (GWT, Dojo, AngularJS), the Actions interface allows
automation without fighting the dreaded Selenium stale element exception agony as items
are constantly added and removed from the DOM. 

Not implementing the Actions interface in Safari is a significant shortcoming for my
team and application. 

We're not concerned that it requires actions to be implemented via javascript and that
they aren't native browser events. We fight the same problems in the other browser
implementations as well. It's a necessary evil. 

Reported by wpballard on 2015-07-27 18:53:46

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

I encountered the same issue as Issue 6615, which has been merged to this issue, looking
for workaround. Thanks.
https://code.google.com/p/selenium/issues/detail?id=6615

Reported by xingjing2006 on 2015-09-01 03:28:26

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Actions action = new Actions(driver);
action.moveToElement(element);
action.clickAndHold();
action.moveByOffset(100,200).moveByOffset(-10, -20).moveByOffset(-10, -10).moveByOffset(-10,
-20).build().perform();
action.release().build().perform();

can you please provide the work around for the above code.

Reported by sreekanth.appam on 2015-09-01 09:03:50

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

For comment 51, take a look at the 2nd code snippet for mousemove under: https://ynot408.wordpress.com/2011/09/22/drag-and-drop-using-selenium-webdriver/

that's a starting point for crafting a JS only workaround.

Reported by daluu@mistsys.com on 2015-09-08 23:53:34

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Reported by luke.semerau on 2015-09-17 17:44:49

  • Labels added: Restrict-AddIssueComment-Commit

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant