Introduction

We've provided several tools to help you try out Jaql. Below, we list the software requirements, describe how to get the Jaql, show how to run the Jaql shell, evaluate examples from the documentatation, and run unit tests.

Requirements

• Java 1.6 or greater
• Cygwin if using MS Windows
• If running your own, distributed cluster, Hadoop 0.17.1, Hadoop 0.18.1, Hadoop 0.18.3, and Hadoop 0.19.0

Jaql Shell

The Jaql shell is a very simple way to run Jaql queries. Currently, it can run with a Hadoop MiniCluster or your own cluster.

To get up and running, you can either download a binary or build from source.

Using a binary

1. Download the tar (for Hadoop 0.18.3 clusters).
2. untar jaql-0.4_hadoop-0.18.3.tgz into DIR

Using the source

Follow these instructions to build Jaql.

Running

1. setup the environment: export JAQL_HOME=DIR/jaql-0.4
2. run it: from DIR/jaql-0.4, run ./bin/jaqlshell. You should see the jaql> prompt if successful.

Note: If you already have a Hadoop cluster installed, when running jaql in local mode, be sure to unset Hadoop related environment variables (e.g., HADOOP_HOME, HADOOP_CONF_DIR, ...).

By default, jaqlshell launches a Hadoop MiniCluster. A MiniCluster runs HDFS, map-reduce, and HBase as multiple threads in a single process. If you already have a distributed cluster installed and what to try out Jaql, pass in the --cluster flag as follows: jaqlshell --cluster.

JaqlShell supports the following options:

jaqlshell [options] [<file1> [<file2> ...]]
options
  -h (--help,-?)             print this message
  -j (--jars) <args>         comma-separated list of jar files to include user
                             defined expressions or data stores
  -b (--batch)               run in batch mode (i.e., do not read from stdin)
  -e (--eval) <expr>         evaluate Jaql expression
  Cluster options
    -c (--cluster)           use existing cluster (i.e., do not launch a
                             mini-cluster)
    -d (--hdfs-dir) <arg>    mini-cluster option: root HDFs directory
    -n (--no-nodes) <arg>    mini-cluster option: number of nodes to spawn
  <file> [<file> ...]        list of input files

Examples

All examples from the documentation can be found in DIR/jaql-0.4/docs/examples.txt. JaqlShell can be used to evaluate the examples either interactively or as a script. To run interactively, fire up the shell, then cut-and-paste. To run as a script, start the shell in batch mode and supply the example file as an argument, i.e., ./bin/jaqlshell -b docs/examples.txt.

Note: Queries from the documentation that intentionally produce exceptions were excluded from examples.txt.

When extending Jaql, be sure to specify the jar file that includes your extensions to the shell. For example, if your extension code is in myextensions.jar, specify the jar file to the shell as follows: ./bin/jaqlshell --jars myextensions.jar.

Unit Tests

The "tests" directory includes other tests that can be used as additional examples. The tests are included under DIR/jaql/tests. Each test includes a Queries file (i.e., DIR/jaql-0.4/tests/coreQueries.txt) and a Gold file (i.e., DIR/jaql-0.4/tests/coreGold.txt) that lists expected outputs, including when failures are expected. To run all of the tests, from DIR/jaql, run ant test. The test outputs and logs will be placed in DIR/jaql/build/test/cache and DIR/jaql-0.4/build/test, respectively. To run only one of the tests, from DIR/jaql-0.4, run ant test -Dtest.include="TestCore" for example. The tests can also be run on a distributed cluster as follows: ant test-cluster.