Export to GitHub

jaql - issue #8

Allow long running Exprs to report status to the system


Posted on Mar 12, 2009 by Grumpy Monkey

Long running functions can timeout during map/reduce when one record is processed for a long time. We should provide a way that any Expr can report that progress is being made. This probably involves placing the hadoop reporter inside the jaql runtime context, but we will probably want to abstract away from hadoop a little bit so it works in serial code as well.

This feature should consider a larger feature of reporting on the status of a query, even when multiple map/reduce jobs are running concurrently. It should also include reporting of things besides progress, eg, number of exceptions that were handled.

Status: Accepted

Labels:
Type-Enhancement Priority-Medium