My favorites | Sign in
Project Home Downloads Wiki Issues Source
READ-ONLY: This project has been archived. For more information see this post.
Search
for
ProgrammingGuideContinuous  
SSS Mapreduce Programming Guide - Continuous Mapreduce
ja , en
Updated Feb 27, 2013 by tatsuhik...@gmail.com

Continuous Mapreduce

SSS Mapreduce provides the function named "Continuous MapReduce" for the continuous execution of jobs. The "Continuous MapReduce" is constituted by the following two functions.

  • The periodical execution of job tree
  • The destructive reading

The periodical execution of job tree

When the jobs are invoked with JobEngine#contExec method instead JobEngine#exec method, SSS Mapreduce invokes the jobs periodicaly. SSS Mapreduce executes the job tree periodicaly until JobEngine#stopContExec is called. And the jobs execution is controled by the client, therefore the jobs execution will stop if the client finishes.

By default, the jobs is invoked every 1000 milliseconds. Use JobEngine#setInterval to change interval of the jobs execution.

public void setInterval(long interval);

The argument of this method is interval by milliseconds.

The destructive reading

The job of "Continuous Mapreduce" reads data from same TupleGroup periodicaly. Therefore, the job reads data read in the previous execution. Thus SSS Mapreduce provides the functions named "the destructive reading".

If the destructive reading is enabled, when tuples are reads SSS Mapreduce removes the tuples in storage servers. In its way, you can prevent to read same tuples serveral times.

Set true to "destructive" flags on creating TupleGroup with GroupID#createRandom method to enable that the destructive reading. When "destructive" flags is enabled, the tuples of new TupleGroup are removed from storage servers when the tuples are read.

Powered by Google Project Hosting