Fixed
Status Update
No update yet.
Comments
pj...@gmail.com
Perhaps one way to solve this would be to allow all of the settings you can set with com.google.appengine.api.files.GSFileOptions.GSFileOptionsBuilder on the com.google.appengine.api.blobstore.UploadOptions class. So you could specify up front what the filename should be or any metadata you need.
bt...@brandonthomson.com
If you retrieve the BlobInfo for that blob key you should be able to get the filename.
ti...@gmail.com
I can get the BlobInfo for the key, but it just gives me the user's upload filename which is not what I need. I ran a test in prod and it shows this:
BlobInfo info = new BlobInfoFactory().loadBlobInfo(blobKey);
// This just gives the user's filename like "cute-kitten.jpg":
log.info (info.getFilename());
However, I'm looking for the autogenerated filename that is created in google storage. I refreshed the cloud storage explorer and found the GS filename generated for this test was "L2FwcHMtdXBsb2FkL2Jsb2JzL2UzNUFBVE0tMEIxM1dwREFTc0t3RW9LOFE" and I can't find a way to get this.
BlobInfo info = new BlobInfoFactory().loadBlobInfo(blobKey);
// This just gives the user's filename like "cute-kitten.jpg":
However, I'm looking for the autogenerated filename that is created in google storage. I refreshed the cloud storage explorer and found the GS filename generated for this test was "L2FwcHMtdXBsb2FkL2Jsb2JzL2UzNUFBVE0tMEIxM1dwREFTc0t3RW9LOFE" and I can't find a way to get this.
ku...@gmail.com
This issue is not specific to Java. I'm using Python and cannot retrieve the Google storage filename after a user upload. The BlobInfo key is unrelated as far as I know (Strangely it starts with "?ISO-8859-1...").
ti...@gmail.com
Facing the same issue. Without knowing GS filename this entire API is useless for GAE app. Its possible to upload data to GS but impossible to read/delete them. Is there any workaround for this issue?
sn...@gmail.com
da...@google.com
em...@pierre-olivier.latour.name
I'm not sure the person posting the last comment actually read the issue. The problem is that we *only* have the blobkey. That's not enough for some use cases using GCS.
For example, it is impossible to add metadata headers to the uploaded files that are stored in GCS.
For example, it is impossible to add metadata headers to the uploaded files that are stored in GCS.
in...@gtempaccount.com
I understand & acknowldge "it is impossible to add metadata headers to the uploaded files that are stored in GCS."
I was responding to
I was responding to
mb...@gmail.com
Ok, that makes sense and I see where you are coming from.
A few clarifications... Just trying to help out the next guy that hits this, since I've since moved from GAE to AWS...
1. The documentation is unclear if you call delete on a blobkey if it deletes the key/pointer or the actual file in GCS. For some reason I assumed that the delete removed the blobkey, but not the actual file.
2. When our application was written to work with GCS, we had a custom uploader that just put files in GCS based on a naming convention into the filestore. So to read the file we didn't have to know a "key" because it was just a user's ID and some other derived info.
When we switched to the official uploader that always returned a blobkey, we couldn't do that anymore. Without changing our code to store the blobkey we would have no way to read it. Perhaps the poster from
A few clarifications... Just trying to help out the next guy that hits this, since I've since moved from GAE to AWS...
1. The documentation is unclear if you call delete on a blobkey if it deletes the key/pointer or the actual file in GCS. For some reason I assumed that the delete removed the blobkey, but not the actual file.
2. When our application was written to work with GCS, we had a custom uploader that just put files in GCS based on a naming convention into the filestore. So to read the file we didn't have to know a "key" because it was just a user's ID and some other derived info.
When we switched to the official uploader that always returned a blobkey, we couldn't do that anymore. Without changing our code to store the blobkey we would have no way to read it. Perhaps the poster from
mb...@gmail.com
I meant that its impossible to access data that resides in GS (are they?) via GS REST API, since we don't know data url. I.e. if I upload data via GAE API I forced to use only this API (e.g. blobstoreService.serve() to give blob data to the user etc.).
However, this issue also adds a lot of confusion. Files are being uploaded to the GS (I can see them in GS Manager) but also appear in GAE app blobstore viewer. Some question arise here:
- will I be charged twice, for GAE app storage and GS storage space
- does it differ from the case when I put data to GS via curl -X PUT http://.../bucker/file in any way, i.e. is GS differs in any way by implementation from Blobstore, or its just a rebrended storage with new REST API to access it and different data namespace?
However, this issue also adds a lot of confusion. Files are being uploaded to the GS (I can see them in GS Manager) but also appear in GAE app blobstore viewer. Some question arise here:
- will I be charged twice, for GAE app storage and GS storage space
- does it differ from the case when I put data to GS via curl -X PUT http://.../bucker/file in any way, i.e. is GS differs in any way by implementation from Blobstore, or its just a rebrended storage with new REST API to access it and different data namespace?
as...@gmail.com
with 1.7.5 release is there any way to set the filename when calling createUploadUrl(java.lang.String, com.google.appengine.api.blobstore.UploadOptions)
Is it correct then, that to do named filed upload I have to use GCS rest api?
Is it correct then, that to do named filed upload I have to use GCS rest api?
cr...@gmail.com
[Comment deleted]
br...@gmail.com
I'm having the same issue with python Blobstore API and GCS. What I found is that, when using blobstore.creat_upload_url('/foo/bar/', gs_bucket_name='foobucket') to generate a upload url just like using blobstore only. It works, the data will be uploaded to GCS, but also leave an instance that you can see in the BlobViewer from application dashboard. So the first question is, does Google charge for both object (blob and gcs)?
Second, when using blobstore.create_gs_key('/gs/bucket/filename') returns a string, I guess it's a key associated with the file exsiting on GCS but it's not documented how to use it. First, I thought it is used to retrieve/delete object in GCS, but it's not! I tried blobstore.delete('key') to delete object in GCS it doesn't do anything. Meanwhile, when using get_uploads('fieldname')[0] to retrieve a Blobinfo from form, you can get BlobKey from this, i.g.:
uploaded_file = self.get_uploads('fieldname')[0]
file_blobkey = uploaded_file.key()
Then I can use this blobkey to delete object in BlobViewer and GCS, then the second question is what does create_gs_key() do? How we gonna use it?
Third question is, how do I serve files from GCS through AppEngine without using AppEngine's bandwidth? I know ImageService can serve image, does it work for pdf/csv/word etc. as well?
Need help here!
Thanks
Second, when using blobstore.create_gs_key('/gs/bucket/filename') returns a string, I guess it's a key associated with the file exsiting on GCS but it's not documented how to use it. First, I thought it is used to retrieve/delete object in GCS, but it's not! I tried blobstore.delete('key') to delete object in GCS it doesn't do anything. Meanwhile, when using get_uploads('fieldname')[0] to retrieve a Blobinfo from form, you can get BlobKey from this, i.g.:
uploaded_file = self.get_uploads('fieldname')[0]
file_blobkey = uploaded_file.key()
Then I can use this blobkey to delete object in BlobViewer and GCS, then the second question is what does create_gs_key() do? How we gonna use it?
Third question is, how do I serve files from GCS through AppEngine without using AppEngine's bandwidth? I know ImageService can serve image, does it work for pdf/csv/word etc. as well?
Need help here!
Thanks
kd...@gmail.com
Or, you could symlink the pycrypto install path to the location of your code:
ln -s /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Crypto/ .
(OS X 10.8)
ln -s /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Crypto/ .
(OS X 10.8)
am...@google.com
am...@google.com
This is fixed in 1.7.4. However, you must use easy_install -Z (--always-unzip) to install PyCrypto. The default zipfile option in OSX 10.8 is incompatible with the sandbox emulation in the dev_appserver.
ap...@gmail.com
Same problem i have while installing the module pycrypto-3.3 in windows 7.
i tried every effort but still i m having the same problem. can anyone suggest me the solution. the error is as :
Traceback (most recent call last):
File "C:\Users\Apurva\Documents\Python\first.py", line 2, in <module>
from Crypto.PublicKey import RSA
ImportError: No module named 'Crypto'
i tried every effort but still i m having the same problem. can anyone suggest me the solution. the error is as :
Traceback (most recent call last):
File "C:\Users\Apurva\Documents\Python\first.py", line 2, in <module>
from Crypto.PublicKey import RSA
ImportError: No module named 'Crypto'
ya...@gmail.com
I am having the same problem. The previous workarounds wouldn't work because dev_appserver.py no longer exist and the structure of how that is done is change. I can use all other 3rd party libraries other than this. Using easy_install didn't help in this case either. I am on Windows platform. Is there anyone that came up with a solution for this? Thanks.
[Deleted User] <[Deleted User]>
Bump, this is still not working for me.
Does anyone have a way of making Crypto work in the local sandbox? Did I miss something? I can't seem to make it work locally.
Does anyone have a way of making Crypto work in the local sandbox? Did I miss something? I can't seem to make it work locally.
rb...@gmail.com
This is still an issue in OS X 10.10.3, App Engine 1.9.23.
For me
For me
de...@gmail.com
This issue is very much relevant even now (Nov 2015) and I just observed it on my Ubuntu 15.10 / Google App Engine for Python 1.9.8
Thanks, I hope there is soon a solution to this.
Thanks, I hope there is soon a solution to this.
Description
site-packages. A natural tendency of lazy developers is to run
easy_install pycrypto. This installs the Crypto module just fine but the
dev_appserver.py still raises an ImportError which is a bug.
For *users*: here is a temporary workaround:
# delete the egg:
sudo rm -fr
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/pycrypto-2.0.1-py2.5-macosx-10.3-i386.egg/
# install it the slightly-less-easy way:
wget
tar -xzf pycrypto-2.0.1.tar.gz
cd pycrypto-2.0.1
python setup.py build
sudo python setup.py install
If you want to consider fixing this for the SDK, it should be pretty easy
to do. Here are a few lines of code you can add to dev_appserver.py around
line 819 just before ALLOWED_SITE_PACKAGE_FILES
try:
import Crypto as _CryptoTest
# I.E. pycrypto-2.0.1-py2.5-macosx-10.3-i386.egg/Crypto
_CryptoBase = os.path.dirname(_CryptoTest.__file__).replace(
os.path.join(os.path.dirname(os.__file__), 'site-packages'),
"")[1:] # removes preceding slash
except ImportError:
_CryptoBase = "Crypto"
Then, the subsequent hard-coded strings "Crypto" can be replaced with
_CryptoBase. I would not recommend using pkg_resources.require('pycrypto')
to find the path since that would introduce setuptools magic.