My favorites | Sign in
Project Home Downloads Issues
New issue   Search
  Advanced search   Search tips   Subscriptions
Issue 158: Adding support for getting body of requests or responses
35 people starred this issue and may be notified of changes. Back to list
Status:  Migrated
Owner:  ----
Closed:  Mar 2013

issue 238

Sign in to add a comment
Reported by, Jul 3, 2011
Adding support for accessing body of requests (useful for http post requests) in the onResourceRequested call back.

Which version of PhantomJS are you using? 1.2

Jul 4, 2011
Project Member #1
I'm not sure how we would encode the content. Typed array comes to mind, but that's not widely supported yet.
Jul 15, 2011
Project Member #2
(No comment was entered for this change.)
Summary: Adding support for getting body of requests or responses
Jul 15, 2011
It seems the body would be part of the "text" property in the content object of the HAR response object:
Jul 21, 2011
After doing some more research, wouldn't it be feasible to add the content of the response to a text property in the data emitted when listening to the NetworkAccessManager::handleFinished and performing a readAll() on the reply argument?  Then taking that byte array and converting to a QTString?

Currently, the signal is coming from:
Then the reply object is:
Where we could call readAll to :

Just wanted to check if I am on the right path here.

Jul 22, 2011
Project Member #5
I'm not sure readAll is wise there. Doing that definitely leaves WebKit with data at all?
Jul 22, 2011
Are you saying that if you call readAll on the response object, it won't leave that data available for WebKit to read?
Jul 25, 2011
Project Member #7
Yes, I am quite sure calling readAll will eat all the received data and leave nothing.
Aug 4, 2011
Off the top of your head, would there then be a way to read a copy of response data?  If you can point me in the right direction, I can start experimenting with it.
Aug 7, 2011
I did a little more research and found out that it might be possible to write a proxy for the QNAM and the NetworkReply.  This way you could capture the data received and still have it accessible after the read has finished.  An example I found pointed me to:

Am I on the right track here?

Aug 7, 2011
Project Member #10
Yes, proxying both the network access manager and network reply is the right approach.
Aug 7, 2011
I'm on my way to figuring this out but have run into a snag.  Keep in mind I'm not that familiar with C++ but after adding in the reply proxy, I get a compile error of: 

Undefined symbols:
  "vtable for NetworkReplyProxy", referenced from:
      NetworkReplyProxy::NetworkReplyProxy(QObject*, QNetworkReply*)in networkaccessmanager.o

Here is my latest gist of the NetworkAccessManager:

Any help would be greatly appreciated!
Aug 14, 2011
Pull request:

We'll definitely need to go through a code review on this one :)
Aug 21, 2011
Project Member #13

We need an option to enable and disable (by default) this data gathering (since it introduces some performance penalty).
Status: Accepted
Labels: Milestone-Release1.3
Sep 14, 2011
Project Member #14
Hmm, some ownership issue and/or race condition in the proxy might provoke a crash.
Sep 15, 2011
Ariya, I'm sorry, but I'm not following.  Can you elaborate a little more?

Also, when I get some time, I can add in a patch to disable the use of the proxy.
Sep 15, 2011
Project Member #16
I fixed the potential crash in
Sep 15, 2011
Sep 15, 2011
Project Member #18
Closing this one as it is implemented already.

For the API/settings improvement, head to  issue 236 .
Status: Fixed
Sep 18, 2011
Project Member #19
Reopened. Likely needs to be rescheduled soon.

It causes regression, see  issue 238 .
Status: Accepted
Sep 18, 2011
Project Member #20
(No comment was entered for this change.)
Blocking: 238
Sep 19, 2011
Project Member #21
I can't find a quick way to resolve this. Probably I will revert it for the time being.
Sep 20, 2011
Project Member #22
Unfortunately I have to revert this, see

We need to have a better working solution for 1.4.
Labels: -Milestone-Release1.3 Milestone-Release1.4
Oct 3, 2011
Ariya, do you have any other information on what needs to be added to the proxy so we can get this back in?  I'd like to be able to help and get this resolved.
Oct 4, 2011
Project Member #24
I have not investigated further. Basically the fix should not regress netlog and netsniff examples (easy to test). See also  issue 238 .
Oct 4, 2011
Sounds good.  I will try to look into this and see if I can find a solution.  I will also look into adding the flag to disable the proxy for performance reasons.
Dec 18, 2011
Project Member #26
Not enough time to resolve for 1.4. Postpone to 1.5.
Labels: -Milestone-Release1.4 Milestone-Release1.5
Mar 8, 2012
Project Member #27
No activity, reschedule.
Labels: -Milestone-Release1.5 Milestone-FutureRelease
Mar 12, 2012
Project Member #28
 Issue 422  has been merged into this issue.
Apr 14, 2012
Project Member #29
 Issue 501  has been merged into this issue.
May 2, 2012
I think, the embedded phantomjs server can act as a proxy which intersects and saves the content of the dowloading files and the sending POST.
The only problem is its unability to parse queries like "GET HTTP/1.1".
The server responses with "Error 400: Bad Request, Cannot parse HTTP request: [GET]".
It seems that fixing the server would be an easier and faster solution for the people wanting to access the bodies of requests and responses.
May 2, 2012
I do have a commit that adds the ability to read the responses @ However there are 2 other issues I know of that need to be fixed in order to get this accepted:

1.  Issue #238  - REGRESSION: netsniff.js does not produce HAR dump
2.  Issue #236  - Settings to control network capture

I'm pretty slammed right now so if someone else is interested in helping out, I'd appreciate it :)
May 8, 2012
#32 Pavel.evst
I made simple solution for that,
In my fork page.onResourceRequested accept hash with postData if its POST-request

Why you need proxy for that? I think its quite simple
May 12, 2012
jgornik, there is one more issue with NetworkProxy approach:
synchronious XMLHttpRequest doesnt work (but asynchronious still)

May 14, 2012
Project Member #34
I would want the resource response text to be not just readable but writable as well (so I can modify a particular resource's response body for instrumentation purposes). See for more info.
Feb 5, 2013
For what it's worth, I have a workaround that I've been using. I followed phantomjs's code into the QTNetworking core and found where the cache is being saved and how the names are hashed. I replicated that using phantomjs and then pulled the content from the QT's cache file. I only tested this with images since that's all I need it for, there will definitely be cases where text documents don't work with it, but for what it's worth, here is my code:
Feb 6, 2013
Project Member #36
Interesting idea, thanks for sharing. Unfortunately this approach also breaks down when the responding server indicates (via HTTP response headers) that the requested resources cannot be cached.
Mar 15, 2013
Project Member #37
Closing. This issue has been moved to GitHub:
Status: Migrated
Sign in to add a comment

Powered by Google Project Hosting