|
Builtin_functions
List of JAQL built-in functions
pragmaconst()Description This is a pragma function to force const evaluation. Parameters (1) inline()Description This is a pragma function to force let/global inlining. Parameters (1) systemexec()Parameters (2) R()Description A very rough cut at calling R from Jaql.
R(string init, string fn, item arg1, ..., item argN) A single R process is forked per RFn instance (i.e., call site in the query). The R process is forked and the init string is passed to R only on the first invocation. To configure R, add -DR.home=<path to R> to the VM arguments. // TODO: need jaql.conf Parameters (2 - ...) corecombine()Parameters (2) compare()Parameters (2) groupCombine()Description groupCombine(input $X, initialFn, partialFn, finalFn) => $YinitialFn = fn($k,$X) e1 => $P partialFn = fn($k,$P) => $P finalFn = fn($k,$P) => $Y Parameters (4) help()Parameters (0 - 1) index()Description element(array, index) is the same as array[index], but it captures a simpler case that doesn't use path expressions. array[index] is transformed to use the element function for better performance. Parameters (2) listExtensions()Parameters (0 - 1) mergeContainers()Description Merge a set of arrays into one array in order, or a set of records into one record. Nulls are ignored. Parameters (1 - ...) perf()Parameters (1) perPartition()Parameters (2) range()Parameters (2) tee()Parameters (1 - ...) hadoopmapReduce()Parameters (1) mrAggregate()Parameters (1) readConf()Parameters (1 - 2) ioarrayRead()Parameters (1) file()Description An expression that constructs an I/O descriptor for local file access. Parameters (1 - 2) hbaseDelete()Description hbaseDelete(tableExpr, keyExpr, columnExpr)
string tableName <- eval tableExpr JArray keyValue <- eval keyExpr JArray of Texts <- eval columnExpr Delete all column values for the given tuple identified by the keyExpr in the table defined by tableExpr. Parameters (3) hbaseFetch()Description hbaseFetch( tableExpr, keyExpr, |columnExpr, {timestamp:timestampExpr, numversions:numversionsExpr}|)- Text tableName <- evaluate tableExpr - JArray keysExpr <- evaluate keyExpr -JArray columnExpr <- evaluate columnExpr - LongItem timestamp <- evaluate timestamp - LongItem numVersions <- evaluate numversions
If only keyExpr is specified, a tuple is returned. If keyExpr and numversions are specified, then multiple values may be returned for a column. In this case, the result that is returned is as follows: {key: value, column: [v1, v2, ...], column: [...]} Parameters (2 - 4) hbaseShell()Parameters (1) hbaseWrite()Parameters (2 - 3) hdfs()Description An expression that constructs an I/O descriptor for HDFS file access. Parameters (1 - 2) hdfsRead()Parameters (1 - 2) hdfsShell()Parameters (1) hdfsWrite()Parameters (2 - 3) http()Description An expression that constructs an I/O descriptor for local file access. Parameters (1 - 3) httpGet()Parameters (1 - 3) localRead()Parameters (1) localWrite()Parameters (2) read()Description An expression used for reading data into jaql. It is called as follows: read({type: '...', location: '...', inoptions: {...}}) Parameters (1) readAdapterRegistry()Description Load the registry from the given file
readAdapterRegistry(string filename) returns file name Parameters (1) registerAdapter()Description Register a key, value pair.
registerAdapter({type: string, inOptions: {}, outOptions: {}}) Parameters (1) unregisterAdapter()Description Unregister a key, value pair.
unregisterAdapter(string key) returns key Parameters (1) write()Description An expression used for writing external data. It is called as follows: write({type: '...', location: '...', outoptions: '...', inoptions: '...'}, expr); Parameters (2) writeAdapterRegistry()Description Write the registry to a given file
writeAdapterRegistry(string filename) returns null Parameters (1) registryreadFunctionRegistry()Parameters (1) registerFunction()Parameters (2) writeFunctionRegistry()Parameters (1) arrayappend()Description append($a, $b, ...) ==> unnest [ $a, $b, ... ] NOT when $a or $b are non-array (and non-null), but that's probably an improvement. NOT when $a or $b are null, but the change to unnest to remove nulls will fix that should append(null, null) be null? it would break any unnest definition... Push unnest into ListExpr? Parameters (1 - ...) arrayToRecord()Parameters (2) asArray()Description This function ensures that input returns an array. Parameters (1) columnwise()Parameters (1) deempty()Parameters (1) distinct()Parameters (1) enumerate()Parameters (1) exists()Description exists(null) = null exists([]) = false exists([...]) = true, when the array has at least one element (even a null) exists(...) = true, when the argument is not an array or a null Parameters (1) merge()Description Merge multiple pipes into one pipe in arbitrary order (like SQL's UNION ALL) Parameters (1 - ...) pair()Description pair(A,B) == [A,B] Parameters (2) pairwise()Parameters (2 - ...) removeElement()Parameters (2) replaceElement()Parameters (3) reverse()Parameters (1) rowwise()Parameters (1) shift()Parameters (2 - 3) slice()Parameters (3) toArray()Description If the input is an array or null, return it; else wrap in an array.
if( $x instanceof type [*<*>]? ) $x else [$x] Parameters (1) indexbuildJIndex()Parameters (2) buildLucene()Parameters (3 - 4) keyLookup()Description [ [key,value1] ] -> keyLookup([ [key,value2] ]) ==> [ [key, value1, value2] ]
Build a hash table on the inner key/value pairs (expr[1]). For each key/value in the outer pairs (expr[0])return [key, value1, value2] tuples. The code assumes that the inner keys are unique (or an arbitrary value is kept)//TODO: support duplicates? raise error? If the outer key does not exist in the inner set,null is returned for the inner value. So this is preserving the outer input (left outer join) // TODO: support full outer by finding inner values that didn't join? // TODO:support spilling large inners? Parameters (2) probeJIndex()Parameters (2) probeLucene()Parameters (2 - 3) exprsplit()Parameters (2) splitArr()Parameters (2) schemaschemaof()Parameters (1) typeof()Parameters (1) xmlxmlToJson()Parameters (1) regexregex()Parameters (1 - 2) regexExtract()Parameters (2) regex_match()Parameters (2) regex_spans()Parameters (2) regex_test()Parameters (2) datedate()Parameters (1 - 2) dateMillis()Parameters (1) dateParts()Parameters (1) now()Parameters (0) nildenull()Parameters (1) emptyOnNull()Description emptyOnNull(e) == firstNonNull(e, []) Parameters (1) firstNonNull()Parameters (0 - ...) nullElementOnEmpty()Parameters (1) nullOnEmpty()Parameters (1) onEmpty()Parameters (2) dbjdbc()Parameters (1) aggany()Description Picks any value. If there is at least one non-null values, picks a non-null value. Parameters (1) argmax()Parameters (2) argmin()Parameters (2) array()Parameters (1) avg()Parameters (1) count()Parameters (1) covStats()Description covStats(array x) = sum [1 x1 x2 ... xn] * [1 x1 x2 ... xn]^T= [ count sum(x1) sum(x2) ... sum(xn) ,sum(x1*x1) sum(x1*x2) ... sum(x1*xn) ,sum(x2*x2) ... sum(x2*xn) ,... ,sum(xn*xn) ] Parameters (1) max()Parameters (1) min()Parameters (1) pickN()Parameters (2) singleton()Description Convert a pipe to a simple value:If the pipe is empty then null If the pipe has one item then that item If the pipe has more than one item then error Parameters (1) sum()Parameters (1) vectorSum()Description vectorSum(array x) = [sum(x1), sum(x2), ..., sum(xn)] Parameters (1) topexprtree()Description An internal method that can be used to print the internal tree of expressions in JSON format. Parameters (1) spanspan()Parameters (2) span_contains()Parameters (2) span_overlaps()Parameters (2) tokenize()Parameters (1) numericabs()Description This function accepts a numbers and returns its absolute value Parameters (1) Examplesjaql> abs(-2); 2 jaql> abs(2-10); 8 div()Description this is a div function Parameters (2) exp()Description raise base of natural log (e) to arg: e^a pow(x,y) = exp( y * ln(x) ) Parameters (1) int()Parameters (1) ln()Description natural logarithm Parameters (1) mod()Parameters (2) number()Parameters (1) number()Parameters (1) pow()Description raise a number to power Parameters (2) toDouble()Parameters (1) stringendsWith()Parameters (2) serialize()Parameters (1) startsWith()Parameters (2) strcat()Parameters (0 - ...) strJoin()Description string strJoin(array items, string sep) Build a string that concatentates all the items, adding sep between each item. Nulls are removed, without any separator. If you want nulls, use firstNonNull(e,'how nulls appear'). Parameters (2) strSplitN()Description strSplitN(string src, string sep, int n) ==> [string1, string2, ..., stringn] sep is a string of one charater. Parameters (3) substring()Parameters (2 - 3) randomrandomDouble()Parameters (0 - 1) randomLong()Description Vivamus auctor, sapien eu pulvinar ultricies, ligula dolor blandit mauris, non egestas est mauris ac tellus. Vivamus semper consequat congue. Donec a odio eget nunc dapibus faucibus. Nam dictum massa sed quam congue vitae rutrum velit mattis. Donec pharetra augue sit amet elit porttitor tristique. Maecenas felis turpis, aliquet quis venenatis id, lobortis non enim. Integer sit amet sodales mi. Donec vitae erat metus, et feugiat tortor. Donec a mi augue, vel convallis nibh. Nulla sit amet convallis tortor. Suspendisse vel quam turpis, eget ornare tortor. Nullam eget sapien turpis. Ut leo justo, congue id adipiscing in, egestas in ipsum. Sed dignissim mi a metus rhoncus dictum. Phasellus egestas aliquam eros, vel mollis lacus consequat consequat. Ut pulvinar, leo vitae bibendum dictum, velit nulla ultricies lacus, id vehicula ante nulla in est. Quisque risus ligula, consequat sit amet aliquam ut, dignissim id justo. Donec interdum velit sed arcu tempus mollis. Morbi diam nulla, auctor cursus sollicitudin nec, auctor quis lectus. Etiam ullamcorper fermentum massa, in commodo lectus auctor feugiat. Sed ac sapien ut sem hendrerit sollicitudin at ut nisi. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Integer nulla erat, mollis eget t Parameters (0 - 1) registerRNG()Parameters (2) sampleRNG()Parameters (1) netjaqlGet()Parameters (1 - 2) recordarity()Parameters (1) fields()Parameters (1) names()Description names($rec) == for $k,$v in $rec return $k == fields($rec)[*][0]; Parameters (1) record()Parameters (1) remap()Parameters (2) removeFields()Parameters (2) renameFields()Parameters (2) replaceFields()Description Replace fields in oldRec with fields in newRec only if the field name exists in oldRec. Unlike remap, this only replaces existing fields. Parameters (2) values()Description values($rec) == for $k,$v in $rec return $v == fields($rec)[*][1]; Parameters (1) |
Sign in to add a comment
