These references omit information on a large body of work that predated your other references by nearly a decade. Random indexing is essentially identical to so-called one-step learning that derived from early work at HNC Software and was refined during my tenure as Chief Scientist at Aptex. The only important difference between random indexing and our earlier work relates to the domain of the original vectors. IN our case, we mostly used vectors sampled from multi-dimensional unit normal distribution, in random indexing one uses ternary or binary vectors. We also experimented with binary vectors, but the hardware of the time favored the continuous representation so we focused on that formulation.

Also, the learning step in random indexing is essentially one iteration of a power law extraction of singular vectors. As typically described, this algorithm cannot be used with more than 2-3 iterations because it collapses onto the dominant eigenvectors. When used for a single iteration, sufficient information from the secondary eigenvectors is retained in the form of the original random initial conditions to avoid problems. It should also be noted that even without the context vector training (i.e. using random vectors with no context training), useful performance can be obtained. These considerations make it clear that random indexing and context vector techniques should be considered as an alternative formulation of LSA and other SVD systems.

Here are some references that you may find interesting:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.87.7893&rep=rep1&type=pdf

http://www.google.com/patents?hl=en&lr=&vid=USPAT5619709&id=4kkhAAAAEBAJ&oi=fnd&dq=William+Caid

http://www.google.com/patents?hl=en&lr=&vid=USPAT5794178&id=kZogAAAAEBAJ&oi=fnd&dq=William+Caid

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VC8-3YMFVB3-1B&_user=7971165&_rdoc=1&_fmt=&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=938841321&_rerunOrigin=scholar.google&_acct=C000050221&_version=1&_urlVersion=0&_userid=7971165&md5=0ac86651fa508bb9b4157b382f281177 http://portal.acm.org/citation.cfm?id=146565.146569

http://www.google.com/patents?hl=en&lr=&vid=USPATAPP10868538&id=L6yfAAAAEBAJ&oi=fnd&dq=William+Caid

http://www.google.com/patents?hl=en&lr=&vid=USPAT6134532&id=J2kGAAAAEBAJ&oi=fnd&dq=William+Caid

http://spiedl.aip.org/getabs/servlet/GetabsServlet?prog=normal&id=PSISDG002606000001000372000001&idtype=cvips&gifs=yes

This paper also uses Semantic Vectors: Newton, G. & A. Callahan & M. Dumontier. 2009. Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library. Second Workshop on Very Large Digital Libraries at the European Conference on Digital Libraries (ECDL) 2009. http://gnewton.ca/u/gn/2009/ecdl2009Newton_20090723.pdf

One more recent paper using Semantic Vectors: Siddhartha Jonnalagadda, Robert Leaman, Trevor Cohen and Graciela Gonzalez. A Distributional Semantics Approach to Simultaneous Recognition of Multiple Classes of Named Entities. CICLing 2010, LNCS 6008 http://www.public.asu.edu/~sjonnal3/home/papers/CIC-LING_60080224.pdf

For earlier (the earliest?) work which models syntax and semantics as a kind of vector product over word vectors see:

Freeman R. J., Example-based Complexity--Syntax and Semantics as the Production of Ad-hoc Arrangements of Examples, Proceedings of the ANLP/NAACL 2000 Workshop on Syntactic and Semantic Complexity in Natural Language Processing Systems, pp. 47-50. ( http://acl.ldc.upenn.edu/W/W00/W00-0108.pdf)

Also:

http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PTXT&s1=Freeman.INNM.&s2="Large+corpus".TI.&OS=IN/Freeman+AND+TTL/

As Ted Dunning points out, non-compositional work is more common. For another opensource project clustering (non-compositional?) senses see also Ted Pedersen's Senseclusters: http://senseclusters.sourceforge.net/

Also Dekang Lin's Web demos:

http://webdocs.cs.ualberta.ca/~lindek/demos.htm