| Sign in

airhead-research

The S-Space package: A collection of algorithms for building Semantic Spaces

Project Home		Downloads		Wiki		Issues		Source

Search

Updated Jun 26, 2009 by David.Jurgens

HyperspaceAnalogueToLanguage

A description of the Hyperspace Analogue to Language implementation of the S-Space package.

Introduction

Hyperspace Analogue to Language (HAL) creates a semantic space from word co-occurrences. A word-by-word matrix is formed with each matrix element is the strength of association between the word represented by the row and the word represented by the column. The user of the algorithm then has the option to drop out low entropy columns from the matrix.

As the text is analyzed, a focus word is placed at the beginning of a ten word window that records which neighboring words are counted as co-occurring. Matrix values are accumulated by weighting the co-occurrence inversely proportional to the distance from the focus word; closer neighboring words are thought to reflect more of the focus word's semantics and so are weighted higher. HAL also records word-ordering information by treating the co-occurrence differently based on whether the neighboring word appeared before or after the focus word.

Typically, the all of the co-occurrence information is used to build semantic vectors are used (for an N x N matrix, these are 2*N in length). However, HAL also offers two possibilities for dimensionality reduction. Not all columns provide equal amount of information that can be used to distinguish the meanings of the words. Specifically, the information theoretic entropy of each column can be calculated as a way of ordering the columns by their importance. Using this ranking, either a fixed number of columns may be retained, or a threshold may be set to filter out low-entropy columns.

For more information on HAL, the following paper is the source of this algorithm:

Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instrumentation, and Computers, 28, 203-208. Available here.

See here for additional papers that use HAL.

HAL Implementation

All of HAL is contained on one file, HAL.java.

"Documents" are given to the algorithm, which allow the user to segment the corpus based on paragraph or sentence boundaries. This has the effect of removing co-occurrence relationships between words on boundaries. Segmentation is not required and is at the users discretion.

Software Requirements

HAL requires Java 6.

Running HAL

HAL can be invoked either using java edu.ucla.sspace.mains.HALMain or through the jar release java -jar hal.jar. Both ways are equivalent.

We provide the following options for changing the behavior of HAL and how the program is run.

Required (at least one of):

-d, --docFile=FILE[,FILE...] a file where each line is a document. This is the preferred input format for large corpora
-f, --fileList=FILE[,FILE...] a list of document files where each file is specified on its own line.

Algorithm Options:

-s, --windowSize=INT how many words to consider in each direction
-r, --retain=INT how many column dimensions to retain in the final word co-occurrence matrix. The retained columns will be those that provide the most information for distinguishing the semantics of the words. Unlike the --threshold option, this specifies a hard limit for how many to retain. This option may not specified at the same time as --threshold
-h, --threshold=DOUBLE the minimum information theoretic entropy a word must have to be retained in the final word co-occurrence matrix. This option may not be used at the same time as --retain.
-W, --weighting=CLASS the fully qualified name of a WeightingFunction implementation. HAL traditionally uses a ramped, linear weighting where those words occurring closets receive more weight, with a linear decrease based on distance.
-F, --tokenFilter=FILE[include|exclude][,FILE...] specifies a list of one or more files to use for filtering the documents. An option flag may be added to each file to specify how the words in the filter filter should be used: include if only the words in the filter file should be retained in the document; exclude if only the words not in the filter file should be retained in the document. The default value is include. An example configuration might look like: --tokenFilter=english-dictionary.txt=include,stop-list.txt=exclude

Program Options:

-o, --outputFormat=text|binary|sparse_text|sparse_binary Specifies the output formatting to use when generating the semantic space (.sspace) file. See SemanticSpaceUtils for format details.
-t, --threads=INT how many threads to use when processing the documents. The default is one per core.
-w, --overwrite=BOOL specifies whether to overwrite the existing output files. The default is true. If set to false, a unique integer is inserted into the file name.
-v, --verbose specifies whether to print runtime information to standard out

The program will produce a .sspace file containing the semantic space information from the input corpus. See FileFormats for exact details on the output formatting.

Sign in to add a comment

Hosted by