From ls...@google.com on September 28, 2010 07:32:20

Please note that 204 No Content, and others have no body either. http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.3 "For response messages, whether or not a message-body is included with
a message is dependent on both the request method and the response
status code (section 6.1.1). All responses to the HEAD request method
MUST NOT include a message-body, even though the presence of entity-
header fields might lead one to believe they do. All 1xx
(informational), 204 (no content), and 304 (not modified) responses
MUST NOT include a message-body. All other responses do include a
message-body, although it MAY be of zero length."

From andrewf...@gmail.com on September 28, 2010 10:45:55

Ok. The current issue is being caused by dpkt reading the rest of the data into the body if the response header has a content-type header, which is wrong according to the RFC. A simple fix is simply to remove those two lines.

Of course, this doesn't solve the larger issue. Making the response intelligently decide whether to parse a body is tricky with dpkt's architecture, because the parsing of the body and headers is done in the http.Message superclass.

From matthaeu...@gmail.com on January 10, 2014 09:11:33

Attached is a patch to fix the issue:

1. Ignore content-type, as it does not say anything about the existence of a body.
2. Do not attempt to read body if response status code is 1xx, 204 or 304.
3. Here comes the ugly part: ignore body if parameter head_response=True has been passed to Response object. HEAD responses are identical to GET responses (including Content-Length), except for the missing body. dpkt can not determine automatically whether this is a HEAD response.

Also attached modified dpkt_http_bug2.zip:

• dpkt_bug.py: call dpkt.http.Response(data, head_response=True)
• stream.txt: file ends with '\r\n\r\n' (instead of '\r\n')

Attachment: http_fix_50.patch dpkt_http_bug2.zip

From matthaeu...@gmail.com on January 12, 2014 04:32:40

Slight improvement: set body consistently to '' (empty string), avoid None.

Attachment: http_fix_50b.patch