My favorites | Sign in
Project Home Issues Source
New issue   Search
  Advanced search   Search tips   Subscriptions
Issue 1197: Consistent input/output/envars access in rule-based functions vs. rule-based commands
1 person starred this issue and may be notified of changes. Back to list
Status:  WontFix
Owner:  ----
Closed:  Dec 2012

Sign in to add a comment
Reported by, Sep 14, 2012

This patch insert a __getitem__ method in the Task class that exposes the same varible substitution patterns as in rule-bases commands.

Eg: the equivalent to "${SRC}" in a command would be tsk["SRC"] within python sources


- Rule-based taskgens have a nice way of specifying commands
    eg:   rule = 'cp ${SRC} ${TGT}'
  It's pretty, clear and easy to understand. This is nice, but it relies on external commands and this is a snag if your main concern is platform-independence. You quickly want to switch to rule functions.

- Rule functions are the right alternative for 100% pure-python portable tasks. Unfortunately the Task class does not provide such a neat high-level interface to access source and target filenames. Instead it exposes waf's low-level internals (Task & Node classes) which is a real burden to the casual user (and to myself as well ;).

    To implement the above command as a function, one must write:

	def my_rule (tsk):
		shutil.copy (

     With the patch, the following is sufficent:

     	def my_rule (tsk):
		shutil.copy (tsk["SRC"], tsk["DST"])

- A consistent high-level interface across rule-based commands and rule-based functions is a must. It greaty improves the readability and the learning curve. Taskgen users should not have to dig into the internals.

Some design considerations

- Since external commands are run from the build directory (instead of the top directory) I had to replace all occurences of .path_from(bld.bldnode) with .path_from(bld.srcnode). It may not be the best choice when users call Task.exec_command(). Maybe it would be wiser to simply use .abspath()

- ${SRC} and ${TGT} (and environment variables as well)  may produce a list of strings instead of just one result. To keep the notation simple with some determinism I opted to support two calling conventions:
   1. scalar context (by default) expects exactly one string and will raise an exception otherwise
   	eg:	tsk["SRC"]   ->      0  input files  => raise WafError
				     1  input file   => return the filename
				     2+ input files  => raise WafError
   2. list context (when the expression is enclosed in brackets)
   	tsk["[SRC]"]	     ->	     always return a list of filenames (possibly empty)

- I first tried implementing a pattern substitution interface looking like:
	tsk % "${SRC}"
  This was closer to the original but raised some issues about how to deal with multiple sources/targets and with quoting. The __getitem__ notiation is cleaner an simpler.

3.5 KB   View   Download
Sep 14, 2012
Project Member #1
Lots of "eval" calls during the runtime will degrade the performance horribly.
Status: Assigned
Sep 14, 2012
I expect most common use case (like 95% or 99%) will be tsk["SRC"] and tsk["DST"], which are eval-free.

Sep 14, 2012
Other use cases are less essential but I think it's good to provide them to avoid forcing the user digging into API internals (Task & Node) when he just has one or two such cases to implement in his project.

I can think of a possible way cope smartly with performance drops. Increase a counter each time an eval() is done. If the counter reach a given threshold (number and/or rate), then it issues a Warning message in an atexit routine, prompting the user to consider some optimisations (with a pointer to the right section in the documentation).

Oct 8, 2012
now I am questionning whether this API change is needed. I was not aware of some points:

- FileHandle inheritance issues on win32. This patch makes a easy to write open(tsk["SRC"]) instead of tsk.inputs[0].read()

- I was not aware of and Node.write() (and wrote something like open(tsk.inputs[0].abspath())) 

- The Node class had improvements since 1.15, it is now easier to use

Note that there is some overlap with #1208 (about environment variables) 
Dec 28, 2012
Project Member #5
I see no reason for this.
Status: WontFix
Sign in to add a comment

Powered by Google Project Hosting