Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for gzip media content on resumable media upload #705

Closed
wonderfly opened this issue Jan 10, 2015 · 9 comments
Closed

Support for gzip media content on resumable media upload #705

wonderfly opened this issue Jan 10, 2015 · 9 comments
Assignees
Labels
imported priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@wonderfly
Copy link
Contributor

From yan...@google.com on December 20, 2012 08:55:05

External references, such as a standards document, or specification? http://javadoc.google-api-java-client.googlecode.com/hg/1.12.0-beta/com/google/api/client/googleapis/media/MediaHttpUploader.html Java environments (e.g. Java 6, Android 2.3, App Engine, or All)? All Please describe the feature requested. Currently we do not gzip media content chunks on resumable media upload. To be consistent, we should gzip it. In 1.13 we had to disable that functionality because the media content length wasn't being computed correctly. So we should fix that.

Original issue: http://code.google.com/p/google-api-java-client/issues/detail?id=691

@wonderfly wonderfly added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. imported priority: p2 Moderately-important priority. Fix may not be included in next release. 2–5 stars labels Jan 10, 2015
@wonderfly wonderfly self-assigned this Jan 10, 2015
@wonderfly
Copy link
Contributor Author

From rmis...@google.com on January 18, 2013 10:27:23

Status: Started

@wonderfly
Copy link
Contributor Author

From rmis...@google.com on January 24, 2013 10:35:28

The problem I am running into is (using example values):

Original content length - 10000
Gzipped content length - 8000
Minimum chunk size - 2000

Initiation request says x-upload-content-length will be 8000

We start uploading a chunk of 2000 from the original content of 10000.
The chunk gets gzipped in HttpRequest and is now 1500.

This 1500 is below the minimum chunk length allowed and an exception is thrown.

We do not know what chunk size to use to get a gzipped chunk that is equal to the specified chunk size.

Cc: yan...@google.com

@wonderfly
Copy link
Contributor Author

From rmis...@google.com on January 24, 2013 10:44:36

Given the above comment I upped the chunk size to MediaHttpUploader.MINIMUM_CHUNK_SIZE * 2 and calculated the gzipped size of each chunk but I still end up with an IOException:

CONFIG: -------------- REQUEST --------------
PUT https://www.googleapis.com/upload/drive/v2/files?uploadType=resumable&upload_id=AEnB2UpVOG2pvLe4WvfDg6h4DbVv2FuaKP89LWVPOiy24dIIPuma3OffEhCqKe1ahYSdHQ16i18nhW8Xty_EpXXXRKtJsNEfrQ Accept-Encoding: gzip
Authorization:
Content-Range: bytes 0-523351/523670
User-Agent: Google-HTTP-Java-Client/1.14.0-beta-SNAPSHOT (gzip)
Content-Type: image/jpeg
Content-Length: 523352

java.io.IOException: insufficient data written
at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.close(HttpURLConnection.java:3120)
at java.util.zip.DeflaterOutputStream.close(DeflaterOutputStream.java:241)
at com.google.api.client.http.GZipEncoding.encode(GZipEncoding.java:38)
at com.google.api.client.http.HttpEncodingStreamingContent.writeTo(HttpEncodingStreamingContent.java:51)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:80)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1084)
at com.google.api.client.googleapis.media.MediaHttpUploader.upload(MediaHttpUploader.java:358)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:438)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:363)

I am not sure what is going on, I expected the above to work.

@wonderfly
Copy link
Contributor Author

From yan...@google.com on January 26, 2013 09:46:27

Doesn't seem worth the effort right now.

Cc: -yan...@google.com
Labels: -Milestone-Version1.14.0 Milestone-Version2.1.0

@wonderfly
Copy link
Contributor Author

From yan...@google.com on February 06, 2013 16:05:10

Labels: -Milestone-Version2.1.0 Milestone-Version1.16.0

@wonderfly
Copy link
Contributor Author

From yan...@google.com on June 10, 2013 06:21:16

Status: Accepted
Owner: pele...@google.com
Cc: ngmic...@google.com
Labels: -Priority-High -Milestone-Version1.16.0 Priority-Medium Milestone-Version1.17.0

@wonderfly
Copy link
Contributor Author

From pele...@google.com on July 28, 2013 23:10:50

Labels: -Milestone-Version1.17.0 Milestone-Version1.18.0

@wonderfly
Copy link
Contributor Author

From yan...@google.com on September 27, 2013 05:05:51

Labels: -Milestone-Version1.18.0

@wonderfly wonderfly removed their assignment May 19, 2016
@dalbani
Copy link

dalbani commented Jul 30, 2018

@JustinBeckwith
This is probably not the best place to ask, but that's where I've landed after extensive googling on the matter...
So my question is: is resumable media upload via the Drive v3 endpoint supposed to support gzip'ed chunks?
All I get is 400 Bad Request when I include a Content-Encoding: gzip in the PUT requests.

@JustinBeckwith JustinBeckwith self-assigned this Feb 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
imported priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

3 participants