Skip to content
This repository has been archived by the owner on Nov 29, 2018. It is now read-only.

Python - cannot Upload a file #3812

Closed
lukeis opened this issue Mar 3, 2016 · 25 comments
Closed

Python - cannot Upload a file #3812

lukeis opened this issue Mar 3, 2016 · 25 comments

Comments

@lukeis
Copy link
Member

lukeis commented Mar 3, 2016

Originally reported on Google Code with ID 3812

Here is my test (portion of it)

self.driver.find_element_by_xpath(uploadFileInputElement)
self.driver.find_element_by_xpath(uploadFileInputElement).send_keys(myFilePath)

I have downloaded 2.21.2 because it was said that issue 3736(On python, upload file
is Unrecognized command) is fixed. 

Selenium version:2.21.2
OS: Mac OS X - lion
Browser: FF 10.0.2

the error:
 File "/Library/Python/2.7/site-packages/selenium-2.21.2-py2.7.egg/selenium/webdriver/remote/webelement.py",
line 143, in send_keys
    value = self._upload(local_file)
  File "/Library/Python/2.7/site-packages/selenium-2.21.2-py2.7.egg/selenium/webdriver/remote/webelement.py",
line 221, in _upload
    {'file': base64.encodestring(fp.getvalue())})['value']
  File "/Library/Python/2.7/site-packages/selenium-2.21.2-py2.7.egg/selenium/webdriver/remote/webelement.py",
line 204, in _execute
    return self._parent.execute(command, params)
  File "/Library/Python/2.7/site-packages/selenium-2.21.2-py2.7.egg/selenium/webdriver/remote/webdriver.py",
line 151, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/Library/Python/2.7/site-packages/selenium-2.21.2-py2.7.egg/selenium/webdriver/remote/remote_connection.py",
line 283, in execute
    return self._request(url, method=command_info[0], data=data)
  File "/Library/Python/2.7/site-packages/selenium-2.21.2-py2.7.egg/selenium/webdriver/remote/remote_connection.py",
line 324, in _request
    response = opener.open(request)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
line 394, in open
    response = self._open(req, data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
line 412, in _open
    '_open', req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
line 372, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
line 1199, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
line 1170, in do_open
    r = h.getresponse(buffering=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py",
line 1013, in getresponse
    response.begin()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py",
line 402, in begin
    version, status, reason = self._read_status()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py",
line 366, in _read_status
    raise BadStatusLine(line)
BadStatusLine: ''


Reported by diana.tzinov@beatport.com on 2012-04-26 15:02:37

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Please can you add a test to reproduce the error.

Reported by david.burns@theautomatedtester.co.uk on 2012-04-29 18:39:22

  • Status changed: NeedsClarification
  • Labels added: Lang-Python

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Reported by barancev on 2012-05-09 22:57:58

  • Labels added: Component-WebDriver

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

i am having the same problem. 
selenium version 2.23.0-py2.6.egg-info
windows server 2003 Standalone
Firefox 12.0

i am trying to setup a mock test to reproduce but not having luck atm

Reported by phil.soliz on 2012-06-08 18:05:39

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

i forget to mention that sometimes it works and sometimes it doesnt

Reported by phil.soliz on 2012-06-08 18:15:07

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

i found that when it does work, a status line will come back as 404 but when it doesnt,
no status is returned:

WORKS:
> c:\documents and settings\administrator\desktop\my-test.py(51)_find_id()
-> find.send_keys(send)
(Pdb) c
WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

> c:\documents and settings\administrator\desktop\my-test.py(51)_find_id()
-> find.send_keys(send)
(Pdb) n
WTF:  HTTP/1.1 404 Not Found

WTF:  HTTP/1.1 200 OK


DOES NOT WORK:

> c:\users\xterm\desktop\my-test.py(52)_find_id()
-> form.send_keys(send)
(Pdb) c
WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

WTF:  HTTP/1.1 200 OK

> c:\users\xterm\desktop\my-test.py(52)_find_id()
-> form.send_keys(send)
(Pdb) n
WTF:
BadStatusLine: BadStatusLine('',)
> c:\users\xterm\desktop\my-test.py(52)_find_id()
-> form.send_keys(send)
(Pdb) c
ERROR
WTF:  HTTP/1.1 200 OK



----------------------------------------------------------------------
Ran 1 test in 45.979s

FAILED (errors=1)

C:\Users\xterm\Desktop>


Reported by phil.soliz on 2012-06-11 19:03:42

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

I can confirm that this issue exists with this configuration:
- Selenium server standalone 2.24.1 (java 1.7.0_0.2 x86_64)
- Selenium python 2.24.0
- Python 2.7.3 x86_64
- Firefox 10.0.2
- Windows 7 64-bit

Reported by jclcheng on 2012-06-25 19:55:15

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

thanks for confirming that the issues is there for you. Please can you attach a test
case that reproduces it

Reported by david.burns@theautomatedtester.co.uk on 2012-06-25 20:25:25

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Unfortunately I can't attach a specific test case due to sensitive content within our
tests.  But basically, I find the input file element and send the path to a jar file:

import os
from selenium import webdriver
browser = webdriver.Firefox()
browser.get("http://127.0.0.1")
browser.find_element_by_xpath("//input[@type='file']").send_keys(os.path.abspath("foo.jar"))

I have just verified that this works for some jar files, but fails for other jar files
with the aforementioned traceback.  I have verified that the files are valid jars.
 Thanks.

Reported by jclcheng on 2012-06-25 21:33:37

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

If you can create a reduced test case that would be great since at the moment there
isnt much to go on other than "it sometimes returns a badstatuserror"

Reported by david.burns@theautomatedtester.co.uk on 2012-06-25 21:54:35

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Try this:

import os
import sys
from selenium import webdriver

browser = webdriver.Firefox()
browser.get("http://cgi-lib.berkeley.edu/ex/fup.html")
browser.find_element_by_xpath("//input[@type='file']").send_keys(os.path.abspath(sys.argv[1]))


This fails if sys.argv[1] is the jar downloaded from https://repository.sonatype.org/service/local/artifact/maven/redirect?r=central-proxy&g=log4j&a=log4j&v=1.2.17&e=jar

However this will pass if sys.argv[1] is a simple text document.


Reported by jclcheng on 2012-06-25 22:25:56

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

I'm seeing this same behavior on 

Ubuntu 12.04
Python 2.7.3
Firefox 13.0.1
Selenium 2.24.0

It looks like it's related to file size. I'm trying to upload a variety of images and
I only see the problem when trying to upload a file that is larger than 546k. 

However, if I catch the exception and keep trying it eventually 'catches' and the upload
goes through. 

Not sure what that indicates.

Code looks like this:

while True:
    try: 
        picture = driver.find_element_by_id('upload_id')
        print 'Trying to send keys again.'
        picture.send_keys(picture_path)

    except BadStatusLine:
        print 'bad status, trying again'
    else:
        break

Reported by sopalenski on 2012-07-02 19:10:19

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

I've attached a zip file with 2 picture files and a script that demonstrates the issue.

One image file 'works.jpg' has no issues.
The other image file 'fails.jpg' illustrates the issue.

Out of the box, the sample will try and load the failing image.

Let me know if you have any questions.

Reported by sopalenski on 2012-07-02 21:36:57


- _Attachment: [sample.zip](https://storage.googleapis.com/google-code-attachments/selenium/issue-3812/comment-14/sample.zip)_

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

ok, i figured out what my problem was -- i was getting the same problem as badstatusline.

i did some more digging and found that selenium chokes on zip files because it is already
trying to zip up a file:

in webelement.py it reads in a zip file that is already zipped and im not sure why
but it chokes on this line:

return self._execute(Command.UPLOAD_FILE,
                            {'file': base64.encodestring(fp.getvalue())})['value']

so what i did was to add this to _upload to fix my problem:

    def _upload(self, filename):
        if filename.endswith(".zip"):
            return filename
        else:
            fp = StringIO()
            zipped = zipfile.ZipFile(fp, 'w', zipfile.ZIP_DEFLATED)
            zipped.write(filename, os.path.split(filename)[1])
            zipped.close()
            try:
                return self._execute(Command.UPLOAD_FILE,
                            {'file': base64.encodestring(fp.getvalue())})['value']
            except WebDriverException as e:
                if "Unrecognized command: POST" in e.__str__():
                    return filename
                elif "Command not found: POST " in e.__str__():
                    return filename
                elif '{"status":405,"value":["GET","HEAD","DELETE"]}' in e.__str__():
                    return filename
                else:
                    raise e

im not sure if this is the best way to go about the enumeration of file types but it
works for me.

Reported by phil.soliz on 2012-08-01 22:55:18

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

I am having the same issue with selenium 2.25.0, python 2.6.6, centos 6.3. I have got
the exact same traceback as diana.tz...@beatport.com. I think it IS related to the
file size as when I try to upload an image it works. If I try to upload a smaller zipped
file this also works so I think my issue specifically is more related to file size.
I am trying to upload a 960KB file on the machine that's running the browser interaction
using send_keys, the file is being sent to a different machine. I tried to use the
solution of catching the exception and keep trying it until it eventually 'catches'
and the upload goes through but was unsuccessful, so I am not any closer to fixing
the problem.

Reported by sarahaddis3 on 2012-08-09 12:45:59

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Very similar issue with selenium 2.25.0, python 2.7, ubuntu 12.04, Firebox 4.0 (yes,
old version but we need it).

The difference is that if I try to upload file more than (about) 600KB (I have 640KB,
did not checked exact failing size) then webdiver just hangs.
I tried to see in debug mode, but it hangs somewhere deep in the urllib code.

Also I noticed that local file upload in the webelement.py, send_keys always ends up
with "Unrecognized command: POST" error (see #3767) and falls back to typing.

For now I fixed this by commenting out local file upload:
   def send_keys(self, *value):
        """Simulates typing into the element."""
        #local_file = LocalFileDetector.is_local_file(*value)
        #if local_file is not None:
            #value = self._upload(local_file)

        typing = []
        for val in value:
            if isinstance(val, Keys):
                typing.append(val)
            elif isinstance(val, int):
                val = str(val)
                for i in range(len(val)):
                    typing.append(val[i])
            else:
                for i in range(len(val)):
                    typing.append(val[i])
        self._execute(Command.SEND_KEYS_TO_ELEMENT, {'value': typing})


Reported by serebrov on 2012-08-16 14:07:18

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Hi,

i reproduced this problem for selenium 2.25 on Vista+FF configuration using code from
comment #10.

As i can see, the reason is that content of local file is zipped and sent to driver's
command executor even if it's not Remote driver instance. 
So local firefox driver sometimes for big files returns stranga unparsable responce
which leads to exception described in initial comment.

Solution is simple imho - as in java binding - do not use LocalFileDetector for non
Remote driver instances. Please take a look at my patch, if it's good please apply.

P.S.: but another rare issue may occur is sending some file path strings to non file
webelements when use Remote driver - then python and java bindings will zip and send
content of this files to hub. it may leads to OOM for jetty threads and performace
decreasing. of course, it's really rare.


Reported by a.u.savchuk on 2012-08-16 23:40:24

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Patch file is missed, sorry.

Reported by a.u.savchuk on 2012-08-16 23:41:09


- _Attachment: [issue_3812.patch](https://storage.googleapis.com/google-code-attachments/selenium/issue-3812/comment-19/issue_3812.patch)_

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Maybe it is better just move file upload into separate method?
Something like 
  webelement.upload_file(local_path) 
which will upload a file in a new way (and raise exception if it failed).
While send_keys will only do what its name says - send keys into the input field.



Reported by serebrov on 2012-08-20 07:40:54

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Hi,

i'm not sure that it's wd way :)
As it was mentioned somewhere that wd minimizes number of possible operations and encapsulate
all variants internally. So send_keys is single way for entering text into input and
uploading file.

Reported by a.u.savchuk on 2012-08-30 16:03:43

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

This issue was closed by revision r17837.

Reported by david.burns@theautomatedtester.co.uk on 2012-09-20 18:55:24

  • Status changed: Fixed

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Alexandr/David, that's not a reliable patch. Imagine someone subclassed (like me) the
WebDriver class and calls send_keys() from an instance of that subclass.

Why don't you do this instead:
import selenium.webdriver.remote.webdriver
is_remote = isinstance(self.parent, selenium.webdriver.remote.webdriver.WebDriver)

BTW, file upload for remote webdriver is broken for me in 2.26 because of this change.

Reported by jonsibonsi on 2012-11-09 00:24:33

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Hi!

I agree that such situation is not processed well. Sorry, i didn't think about this.

But your checking by isinstance() will return true always due all "local" webdriver
classes are subclasses of remote.

Also i'm not sure that "import ..." will work because of remote.webdriver is already
loaded. Is this code worked for you?

I'll try to find solution. And, i think, it'd be better if you create new issue with
example of broken file upload which uses your way.

Thank you.

Reported by a.u.savchuk on 2012-11-20 01:51:20

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Yes, that "import ..." works for me. It's not already loaded in my webelement.py, and
even if it was it wouldn't have failed. :)

You're right, I just realized all "local" classes are in fact subclasses of the remote
one. In that case why not:

is_remote = self.parent.__class__ is selenium.webdriver.remote.webdriver.WebDriver

Reported by jonsibonsi on 2012-11-20 02:00:57

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Issue 4603 has been merged into this issue.

Reported by david.burns@theautomatedtester.co.uk on 2013-03-24 14:06:13

@lukeis
Copy link
Member Author

lukeis commented Mar 3, 2016

Reported by luke.semerau on 2015-09-17 18:15:17

  • Labels added: Restrict-AddIssueComment-Commit

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant