Summary

This page contains brief instructions for installing and running the prerelease version (0.8) of the Semantic Vectors package from the University of Pittsburgh.

These sketch instructions presume that you're reasonably familiar with Java, Ant, CLASSPATHs, etc. If not, you might struggle a bit. Better documentation will hopefully follow sometime soon.

Prerequisites

• Make sure that you have Apache Lucene (version 2.2.0 or more recent) running, and that the lucene core and lucene demo files are included in your CLASSPATH.

To Install from Jar Distribution

• Download the most recent semanticvectors-*.*.jar distribution from this site.
• Add the file (including full path) to your CLASSPATH.

To Build from Source

• Download the most recent semanticvectors-*.*.tar.gz or semanticvectors-*.*.zip archive and expand the archive (using tar -zxvf or unzip).
• Feel free to check out the source code from svn and build using ant.
• Optionally, you can unpack the jar distribution using jar -xvf semanticvectors-*.*.jar and compile by running ant in the trunk directory.
• Make sure that either the resulting build/classes directory or build/*.jar file is in your CLASSPATH.

To Build and Search a Model

• Create a Lucene index using the Lucene demo, by running java org.apache.lucene.demo.IndexFiles on the directory containing your corpus.
• Create term and document vectors by running java pitt.search.semanticvectors.BuildIndex.
• To search the resulting model, run java pitt.search.semanticvectors.Search QUERYTERMS.
• If the upper case term NOT appears in your QUERYTERMS, the query parser will add the terms preceding the NOT term and negate all the terms after it. See VectorNegation.
• To compare two concepts, run java pitt.search.semanticvectors.CompareTerms "QUERYTERMS1" "QUERYTERMS2".
• For more information on searching, see SearchOptions.

Training Cycles

Models can be built in several phases by passing the document vectors back to rebuild new term vectors. See TrainingCycles.

Bilingual Models

For instructions on building a bilingual model from a parallel corpus, see BilingualModels.

Positional Indexes

For instructions on building an index based on term positions, see PositionalIndexes.

Permutation Search

Indexes can now be built that encode directional relationships between words. See PermutationSearch.

Vector Store Formats

The SemanticVectors package currently (as of version 1.6) supports two different vector store formats, a plain text format and an optimized format created by the Lucene I/O packages. For more information including format translation utilities, see VectorStoreFormats.

Developer API Docs

See http://semanticvectors.googlecode.com/svn/trunk/doc/index.html Some useful information may be found in the ReleaseLog.