Use-env Config File Syntax

The use-env plugin config files (also called environment override files) have a simple, shell-like syntax which allows flexible customization of environment variables on a per-job or even per-node and per-task basis. This page gives an overview of that syntax.

Lines in the use-env config file(s) can have the following format. A '#' anywhere on the line indicates a comment. Statements are separated by newlines or semicolons ";".

Config files can be included from other files with the "include" statement

include name

will include file "name" from the same directory as the file in which the "include" was invoked. An absolute pathname may also be specified, e.g.:

include /etc/slurm/environment/foo

Configuration that should only be applied to remotely executed tasks may be specified in special "in task" blocks, which have the form

in task { statments... }

This block, if present, will be read by each task in the job just before exec() is called. This allows the environment to be tailored for a specific task, for example:

in task { if ($SLURM_PROCID == 0) LD_PRELOAD = "$LD_PRELOAD libdebug.so" endif }

would append libdebug.so to LD_PRELOAD only for task 0 in the job. The rest of the config file is ignored by the task. Likewise, these task blocks are ignored when the config file is parsed by srun (except for syntax checking).

The simplest form of expression in the config file is to assign a new value to an environment variable

identifier = value

Where identifier is a valid environment variable of the form [A-Za-z_][0-9A-Za-Z_] and value can be any arbitrary quoted string or string literal. Environment variables (and other locally defined symbols or keywords) can be expanded by the familiar form of $ID or ${ID}. Variable expansion will occur in both unquoted and quoted strings. Whitespace outside of quoted strings is ignored.

Examples of assignment are

MYENV = foo # Valid: MYENV="foo" MYENV2 = $MYENV/bar # Valid: MYENV2="foo/bar" MYENV3 = ${MYENV}bar # Valid: MYENV3="foobar" MYENV3 = "${MYENV}bar" # Valid: MYENV4="foobar" MYENV4 = foo bar # Invalid MYENV5 = "foo bar" # Valid: MYENV5="foo bar" MYENV6 = "foo \"bar\"" # Valid: MYENV6="foo "bar""

There are additional assignment operators that may be used in assignment expressions as well. These include:

|= Set new value only if identifier was previously unset. += Prepend value to colon-separated identifier (e.g. PATH) =+ Append value to colon-separated identifier (e.g. PATH)

For example:

``` MYENV = foo MYENV |= bar # Does nothing. MYENV="foo"

PATH = /usr/bin # PATH = "/usr/bin" PATH += /bin # PATH = "/bin:/usr/bin" PATH =+ /usr/local/bin # PATH = "/bin:/usr/bin:/usr/local/bin" ```

Note that

PATH += "/usr/bin"

is the same as

PATH = /usr/bin:$PATH

except that when $PATH is empty, the trailing ":" will not appear when using the "+=" operator.

Environment variables may also be unset using the simple unset expression

unset identifier

For example:

unset MYENV

Would unset the environment variable MYENV from the current env.

A small set of keywords are available within the config file which describe current parameters of the executing job. These include

SLURM_JOBID Current SLURM job id SLURM_STEPID Current SLURM job step id SLURM_NNODES Number of nodes in current job SLURM_NPROCS Number of processes in current job SLURM_CMDLINE Remote command line for this job SLURM_ARGC Number of command line arguments SLURM_ARGV* Command line argument(s) ARGV0-ARGVN

In task context, the following additional keywords are also available

SLURM_PROCID Global task id or rank SLURM_NODEID Global node id

These are called keywords because their values cannot be overridden by the user. However, they are referenced just like any other variable.

A full list of keywords and their values can be dumped to stderr with the "dump keywords" command. See the DEBUGGING section below.

DEFINING LOCAL SYMBOLS

Occaisionally it may be desireable to define new variables that are not exported to the current environment. The "define" keyword is used for this purpose

define identifier = value

works much like the assignment expression, except that the variable is not exported to the local environment (and thus, not to the job).

Locally defined variables such as these are undefined with the "undefine" keyword:

undefine n

will delete "n" from the symbol table.

CONDITIONAL EXPRESSIONS

The use-env configuration file supports conditional expressions of the form

if (tests) statements else if (tests) statements else if ... statements else statements endif

Where tests can have combinations of the following forms

``` value < value # Numeric comparison only value > value # Numeric comparison only value >= value # Numeric comparison only value <- value # Numeric comparison only value == value # Numeric or string compare value !- value # Numeric or string compare value # True if var is not 0 or empty string; defined var # True if var is defined S matches P # True if string S matches the glob expression P

! tests tests && tests tests || tests ( tests ) ```

For example:

``` if ($SLURM_NNODES > 100) MORE_THAN_100_NODES = 1 else if ($SLURM_NNODES > 50) MORE_THAN_50_NODES = 1 else FIFTY_NODES_OR_LESS = 1 endif

if ( "$SLURM_ARGV0" matches "myapp") include env.myapp endif ```

DEBUGGING

Other commands that are mainly useful for debugging include:

print "STRING" Print the value of string to stdout set debuglevel N Set the debug level for the parser to value N dump keywords Dump a list of currently defined keywords dump symbols Dump a list of currently defined local symbols dump all Dump both of the above

The use-env plugin also looks for the environment variable:

SPANK_USE_ENV_DEBUG

which will increase the verbosity of debug logs for the use-env parser if non-zero.