My favorites | Sign in
Project Home Downloads Issues
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 132: Passing variables to page for page.evaluate
28 people starred this issue and may be notified of changes. Back to list
Status:  Fixed
Owner:  ----
Closed:  Apr 2012


Sign in to add a comment
 
Reported by roejame...@gmail.com, Jun 19, 2011
There's no current easy way to pass variables from the start script to the page. Currently we can only transport from the page to the start script.

This is a problem because in order to pass, for example, a command line arg to page.evaluate, the only way to do it is to make the anonymous function in page.evaluate a string, and append the variable. It would be much better to be able to just pass the variable to the script, that way the workaround isn't needed.

I haven't thought a lot about this yet, but I imagine something like..

page.open('http://site.com' function(status) {
    var thisvar = 't';
    // as it is now
    page.evaluate('function() {' +
                  'var thisvar = \''+thisvar+'\';' +
                  '}');
    // I propose (can have multiple comma separated args), e.g. page.send(thisvar, anothervar, ..)
    page.send(thisvar);
    page.evaluate(function() {
                      console.log(thisvar);
                  });
});

If that won't work, then we could try passing an object..

var obj = new Object();
obj.thisvar = thisvar;
page.send(obj);
Jun 19, 2011
Project Member #2 ariya.hi...@gmail.com
Alternatively, as argument to the evaluate function, e.g:

  page.evaluate(function(args) {
      console.log(JSON.stringify(args));
  });
Jun 19, 2011
#5 roejame...@gmail.com
Absolutely! Much better idea than the ones I had before. :)

Would this be good?:

  page.evaluate(function() {
      console.log(args[0]);
  }, arg, ...);

It would then be accessible with an args array. Python can do infinite args easily with *args. This *might* be doable in C++ by va_start.

http://www.cplusplus.com/reference/clibrary/cstdarg/va_start/
Jun 19, 2011
#6 roejame...@gmail.com
Actually I don't know how well (Py)Qt would work with that method. But if it's doable it'd be pretty nice.
Jun 19, 2011
Project Member #7 ariya.hi...@gmail.com
Since we only allow primitive object (no function, no closure) as the return value for WebPage.evaluate(), we might as well make it a requirement that the object passed to the evaluate() is the same.

If people need complicated object, they can always use JSON.
Jun 24, 2011
#8 roejame...@gmail.com
(No comment was entered for this change.)
Labels: -Milestone-FutureRelease Milestone-Release1.3
Jun 24, 2011
#9 roejame...@gmail.com
"If people need complicated object, they can always use JSON."

I completely agree. No need to make it anymore complicated than it needs to be. :)
Jun 24, 2011
#10 roejame...@gmail.com
(No comment was entered for this change.)
Owner: roejame...@gmail.com
Jun 28, 2011
#11 hunt...@gmail.com
In relation to this issue, I found myself having to embed the scope of any functions I wanted to call within evaluate within its scope of course i.e.

page.evaluate(function() {
  function a() {
  }

  a();
});

The problem with this approach is that the code can get a little messy - especially if there are a few functions within the evaluation's body. Referencing functions in the outer scope of the evaluation fails of course.

Can anyone think of a way to express functions that are available within the evaluation, but declared in its outer scope? Would others find this a useful feature?
Jun 28, 2011
#12 roejame...@gmail.com
The only workaround seems to be converting the function to a string then passing it in, which would work with our plans.

However, notice that JSON is incompatible with functions, so it has to be turned into a string another way, then passed as a string, and converted from.

  afunc = function() {
    console.log('afunc');
  };
  page.evaluate(function(afunc) {
      afunc = new Function(afunc);
      afunc();
  }, String(afunc));

Hopefully that's good enough for you?
Jun 28, 2011
#13 roejame...@gmail.com
I forgot to note; if I remember right, the above approach doesn't technically work, but should get the idea across. There may be a similar way to do that.
Jul 2, 2011
#14 p...@peterlyons.com
Here's my hack workaround to pass data from phantom.args into the page.evaluate (coffeescript)

    toRun = ->
      $('#id_username').val 'some.email@example.com'
      $('#id_password').val 'PASSWORD'
      $('#login').submit()
    page.evaluate toRun.toString().replace('PASSWORD', phantom.args[0])
Jul 2, 2011
#15 hunt...@gmail.com
Thanks. I'm also wondering if passing in query parameters would be useful in this scenario now...

Sep 7, 2011
Project Member #16 ariya.hi...@gmail.com
This apparently requires a lot of effort and won't be done in time for 1.3 so sadly I have to defer it to 1.4.

Once we get  issue 31  solved, it would be an easier effort.
Labels: -Milestone-Release1.3 Milestone-Release1.4
Sep 12, 2011
#17 voronk...@gmail.com
Here is a fast-dirty hack-wrapper:

evaluateWithVars = function(page, func, vars)
{
	var fstr = func.toString()
	for(var v in vars)
	{
		switch(typeof(vars[v]))
		{
			case "string":
				fstr = fstr.replace("_VARS_"+v, "'"+vars[v]+"'")
				break
			default:
				fstr = fstr.replace("_VARS_"+v, vars[v])
		}
	}
	return page.evaluate(fstr)
}

var page = new WebPage();
page.onConsoleMessage = function (msg) { console.log(msg); };
page.open("about:blank", function(status) {
	var query = "test"
	evaluateWithVars(
		page,
		function() { console.log(_VARS_query) },
		{ "query": query }
	);
})

Does anyone know how to bind evaluateWithVars to WebPage() or override basic WebPage::evaluate()? WebPage.prototype is not working.
Sep 12, 2011
#20 nperria...@gmail.com
For the records, I implemented a similar method in Casper.js:
https://github.com/n1k0/casperjs/blob/master/casper.js#L239
Sep 12, 2011
#22 voronk...@gmail.com
More accurate variant without types dependencies and _VARS_ prefix:

evaluateWithVars = function(page, func, vars)
{
	var fstr = func.toString()
	//console.log(fstr.replace("function () {", "function () {\n"+vstr))
	var evalstr = fstr.replace(
		new RegExp("function \((.*?)\) {"),
		"function $1 {\n" + 
			"var vars = JSON.parse('" + JSON.stringify(vars) + "')\n" +
			"for (var v in vars) window[v] = vars[v]\n" +
		"\n"
	)
	console.log(evalstr)
	return page.evaluate(evalstr)
}

var page = new WebPage();
page.onConsoleMessage = function (msg) { console.log(msg); };
page.open("about:blank", function(status) {
	var query = "test"
	evaluateWithVars(
		page,
		function(sdf) { console.log(query1) },
		{ "query1": [1,2,3, function(a) {console.log(a)}] }
	);
})
Sep 16, 2011
Project Member #23 ariya.hi...@gmail.com
WebPage is just a wrapper, not a "native" JavaScript object. Thus, its prototype is unavaiable/can't be extended.
Sep 28, 2011
#24 roejame...@gmail.com
(No comment was entered for this change.)
Owner: ---
Oct 8, 2011
Project Member #25 ariya.hi...@gmail.com
 Issue 258  has been merged into this issue.
Oct 21, 2011
#26 neli...@gmail.com
Another work-around is to use eval():

eval("function fn() { $('#id').val('" + value + "');}");
page.evaluate(fn);

Dec 1, 2011
#27 jgon...@gmail.com
Another version. It prepends the injected vars with $ but can work without prepending too (it's similar to voronkovm's version but does not pollute window). I just find those vars easier to distinguish with $ before them ;)

It can also inject a function inside another function, so that you can have chains of such functions with injected arguments (I find it useful), e.g.:

var query = new Query(page, 'h1');
console.log(query.text);

var Query = function(page, query) {
  this.query = query;
  this.page = page;
};

Query.prototype = {
  _evaluate: function(fn) {
    return this._page.evaluate(injectArgs({ query: this.query, fn: fn }, function() {
      var element = document.querySelector($query);
      
      return $fn(element);
    }));
  },

  get text() {
    return this._evaluate(injectArgs({ say: 'hello' }, function(element) {
      console.log($say);
      return element.textContent;
    }));
  }
};

injectArgs() implementation:

var injectArgs = function(args, fn) {
  var stringifyArgs = function(argsString) {
    var splittedArgs = argsString.split(',');
    for (var i=0; i<splittedArgs.length; ++i) {
      splittedArgs[i] = JSON.stringify(splittedArgs[i].trim());
    }
    return splittedArgs.join(', ');
  };
  
  var newFn, normalArgs, code = fn.toString();
  
  code = code.replace(/function .*\((.*?)\) {/, function(str, p1) {
    var name, arg, newStr = "";
    normalArgs = p1;
    for (name in args) {
      arg = args[name];
      if (arg instanceof Function) {
        newStr += "var $" + name + " = " + arg.toString() + ";\n";
      } else {
        newStr += "var $" + name + " = " + JSON.stringify(arg) + ";\n";
      }
    }
    return newStr;
  });
  code = code.slice(0, -1);
  if (normalArgs === '') {
    newFn = new Function(code);
  } else {
    newFn = eval('new Function(' + stringifyArgs(normalArgs) + ', code)');
  }
  //console.log(newFn.toString());
  return newFn;
};
Dec 8, 2011
#28 wangyang...@gmail.com
just ran across this post and i'm glad to share my implementation. 
it's more elegant because it's simpler and there's no limit to type or number of parameters.

function evaluate(page, func) {
    var args = [].slice.call(arguments, 2);
    var str = 'function() { return (' + func.toString() + ')(';
    for (var i = 0, l = args.length; i < l; i++) {
        var arg = args[i];
        if (/object|string/.test(typeof arg)) {
            str += 'JSON.parse(\'' + JSON.stringify(arg) + '\'),';
        } else {
            str += arg + ',';
        }
    }
    str = str.replace(/,$/, '); }');
    return page.evaluate(str);
}
Dec 8, 2011
#30 wangyang...@gmail.com
add some comments to previous evaluate() function.

suppose the function is: function funcA(x, y) { ... }
just call it like this:  evaluate(page, funcA, 0, "1");

this is a simple test that i used to verify it.

var page = require('webpage').create();
page.onConsoleMessage = function(msg) {
    console.log(msg);
};
var func = function() {
    console.log('hello, ' + document.title + '\n');
    for (var i = 0, l = arguments.length; i < l; i++) {
        var arg = arguments[i];
        console.log(typeof arg + ':\t' + arg);
    }
};
page.onLoadFinished = function() {
    evaluate(page, func, true, 0, 'string', [0,1,2], {a:0}, function(){}, undefined, null);
    phantom.exit(0);
};
page.open('http://www.google.com/');
Dec 8, 2011
#31 nperria...@gmail.com
For the records I've just added arguments passing for evaluation into CasperJS, I used this little class which will parse the function args and inject the corresponding values from a passed context: https://github.com/n1k0/casperjs/commit/d8d083331dc4ac399fc803b3ce4bfc211c939d89#L0R1652
Dec 8, 2011
#32 jgon...@gmail.com
wangyang, your solution is simple but:

a) doesn't let you pass a function as an argument
b) does not have named arguments which means that you have to remember the number of each argument

nperria, could you provide some use example?
Dec 8, 2011
#33 nperria...@gmail.com
jgon> sure, the example in the commit message should be self explanatory: https://github.com/n1k0/casperjs/commit/d8d083331dc4ac399fc803b3ce4bfc211c939d89
Dec 8, 2011
#34 jgon...@gmail.com
Oh, I didn't scroll up, thanks ;)
Dec 12, 2011
#35 wangyang...@gmail.com
hi jgon, with my implementation in comment 28,

a) you can pass a function as an argument:

page.onConsoleMessage = function(msg) { console.log(msg); }
function f0(x, y) { console.log(typeof x); x(); console.log(typeof y); y(); }
function a0() { console.log('this is function a0 as an argument'); }
function a1() { console.log('this is function a1 as another one'); }
evaluate(page, f0, a0, a1);

b) named arguments are also supported because you can pass objects as arguments:

page.onConsoleMessage = function(msg) { console.log(msg); }
function f0(x) { console.log(typeof x); console.log(x.a0); console.log(x.a1); }
var params = { a0:'i have a name', a1:'i have, too' };
evaluate(page, f0, params);

Dec 13, 2011
#36 jgon...@gmail.com
wangyang, you're right, I didn't read the code carefully enough. Seems like a good solution to me. Will test it later in my code.
Dec 18, 2011
Project Member #37 ariya.hi...@gmail.com
Not enough time to resolve for 1.4. Postpone to 1.5.
Labels: -Milestone-Release1.4 Milestone-Release1.5
Jan 11, 2012
Project Member #38 detroniz...@gmail.com
Hi all,

I have been a bit away, so I'm going through all the past communication.

After having re-read this whole issue/thread, I realise that we all seem to have reached the same conclusion: JSON Objects are enough.

Above here there are many possible solution to this problem (every example has it's own pros and cons). What is to decide is where do we fit this in.

CasperJS, that is now a rolling project on it's own, has decided it's implementation.
How do we proceed here?

I have a proposal in few points:
- we should provide an overloadable "evaluate" method on the "webpage" object
- we should handle var serialization/deserialization ourself
- we should handle "bad" usages gracefully

The this point is very important to me: we have to assume that some people might end up passing a non "plain" JSON object: in the evaluate function we should device a mechanism to extract the "plain" part of an object and provide that to the internal scope.

In alternative, we should discard non plain objects.

What do you think?
Jan 12, 2012
Project Member #39 ariya.hi...@gmail.com
I suggest waiting for  issue 226 . Workarounds to implement the variable passing will break sooner or later.
Labels: Module-WebPage Component-Logic
Jan 13, 2012
Project Member #40 joniscoo...@googlemail.com
FWIW, this is how Poltergeist is doing this (code in CoffeeScript):


  evaluate: (fn, args...) ->
    @native.evaluate("function() { return #{this.stringifyCall(fn, args)} }")

  execute: (fn, args...) ->
    @native.evaluate("function() { #{this.stringifyCall(fn, args)} }")

  stringifyCall: (fn, args) ->
    if args.length == 0
      "(#{fn.toString()})()"
    else
      # The JSON.stringify happens twice because the second time we are essentially
      # escaping the string.
      "(#{fn.toString()}).apply(this, JSON.parse(#{JSON.stringify(JSON.stringify(args))}))"
Jan 17, 2012
#41 theicf...@gmail.com
@Comment 28
Works for me. Thanks!
Jan 19, 2012
#42 markstew...@gmail.com
As touched on in #40, line 7 in #28 should read:

  str += 'JSON.parse(' + JSON.stringify(JSON.stringify(arg)) + '),';

or it'll blow up on apostrophes.
Feb 5, 2012
#43 nonp...@gmail.com
#28 can be simplified using the approach in #40, e.g.

function evaluate(page, func) {
    var args = [].slice.call(arguments, 2);
    var fn = "function() { return (" + func.toString() + ").apply(this, " + JSON.stringify(args) + ");}";
    return page.evaluate(fn);
}

It's then used like this (setting the value of field "q" in the form "f" to a specified value):

var sum = evaluate(page, function(text) {
    document.forms["f"].elements["q"].value = text;
}, "PhantomJS");
Feb 10, 2012
#44 wangyang...@gmail.com
as mark mentioned in #40, apostrophes can break my evaluate function in #28.
thanks to his modification, a bug fix version is as follows:

function evaluate(page, func) {
    var args = [].slice.call(arguments, 2);
    var str = 'function() { return (' + func.toString() + ')(';
    for (var i = 0, l = args.length; i < l; i++) {
        var arg = args[i];
        if (/object|string/.test(typeof arg)) {
            str += 'JSON.parse(' + JSON.stringify(JSON.stringify(arg)) + '),';
        } else {
            str += arg + ',';
        }
    }
    str = str.replace(/,$/, '); }');
    return page.evaluate(str);
}
Feb 10, 2012
#45 wangyang...@gmail.com
hi nonp, the function in #43 doesn't handle function and undefined correctly.
please use my test case in #30 to check it.
Mar 19, 2012
Project Member #46 ariya.hi...@gmail.com
No time for 1.5. Rescheduled.
Labels: -Milestone-Release1.5 Milestone-Release1.6
Mar 23, 2012
#47 rfl109....@gmail.com
W
Le 20 mars 2012 01:14, <phantomjs@googlecode.com> a écrit :
Apr 6, 2012
#48 wangyang...@gmail.com
just pulled a request for this issue.
https://github.com/ariya/phantomjs/pull/231
it's based on #44, with some further modifications, though.
usage: page.evaluate(func[, arg0, arg1, arg2, ...])

maybe it's not a 'once-for-all' solution, but it works.
the only problem is that, you cannot pass an object with another
variable referred by it.
however, i believe 90% of the demand could be satisfied now.
Apr 9, 2012
Project Member #49 ariya.hi...@gmail.com
I'd like to have this implemented as close to the metal as possible (possible right in the layer after the JavaScript engine). However, looking at the pull request above, I see that this implementation is already very good and should cover most of the cases.

I'll do some testing and if no serious regression is found, I'll likely merge it. Thanks a lot!
Apr 9, 2012
Project Member #50 detroniz...@gmail.com
I reviewed the code and I made some very small remarks.
Code style stuff, so I'm sure he can still iron those out before merging.
Apr 9, 2012
#51 wangyang...@gmail.com
hi detro,

just fixed them and updated. many thanks for your comments!
Apr 14, 2012
Project Member #52 ariya.hi...@gmail.com
Danny's implementation is landed in https://github.com/ariya/phantomjs/commit/81794f9096. Thanks, Danny!
Status: Fixed
Mar 15, 2013
Project Member #53 james.m....@gmail.com
This issue has been moved to GitHub: https://github.com/ariya/phantomjs/issues/10132
Sign in to add a comment

Powered by Google Project Hosting