|
Project Information
Links
|
enqueue (ENQ) - rough draft ideaTasks can be enqueued and have a status, they can consist of ordered list of steps which are themselves Tasks. Workers pick up batches of tasks and process all the steps in a batch until a blocking step is reached, then the worker puts the task back in the queue . (tasks will only be elligible to be fetched from the queue when the blocking conditions pass, or when their overloaded Ready method returns true. NB Todo: write up notes comparing this approach to the Rabbit MQ examples, (e.g. https://github.com/rabbitmq/rabbitmq-tutorials/blob/master/dotnet/Worker.cs) and then comment on the differences and where to draw the line of when to use which? i.e. what this project is not for. One immediate difference is that this project does not queue a message for delivery, it queues the processing of a step of work. This is much more like a cross between a very simple queue and a simplified workflow engine? (need examples.) Update 31.10.2012 - I'm busy looking into renting a cloud server (win 2008 web R2) to use as a team city build server for the project. My company (goblinfactory) will be sponsoring the build server for ENQ. Update 1.11.2012 -Dedicated Windows 2008 R2 build server bought and paid for, now busy setting up Team City. traditional blocking codeexample of an online insurance quote application
// psuedo code
public Quote[] GetQuotes( ... ) {
var quote1 = SlowWebserviceCallBlockUntilReply( ... ); // should return within 30 seconds
var quote2trackingNo = SubmitViaSlowWebServiceCall2( ... );
// wait a maximum of 4 hours for supplier 2 to email us back quote 2, check email every 5 minutes
Quote quote2 = null;
while( quote2 = readEmail(quote2trackingNo).ParseQuote2() == null )
{
thread.Sleep(5 minutes));
if (wait > 4 hours ) quote2 = Quote.NoReply;
}
var bestQuote = Compare(quote1,quote2);
if (bestQuote.commission < minimum)
{
EmailQuoteToSupervisor(bestQuote, ... );
}
psuedo "queued" code
// _"Task" in the code below is custom pseudo Task class_
Func<ParentTask,StepTask,bool> Ready { get; set;}
Func<ParentTask,StepTask,bool> Timeout { get; set; }
Func<ParentTask,StepTask,int, bool>[ ] Steps { get; set; }
var task = new GetQuoteTask(
Ready : (task,step) => { return true;},
Timeout : (task,step) =>{ return (task.Elapsed > TimeSpan.FromDays(2))},
Steps:
new Task(Ready:true, Timeout : TimeSpan.FromSeconds(30)
task.Steps[2].enqueue(); // run task 2 in parallel, tasks are run sequentially by default
task.Quote1 =await SlowWebserviceCallBlockUntilReply ( ... );
},
(task,step, 2) =>{
task.Quote2 = await
},
);
task.Enqueue(..);
an alternative way of writing these tasks might be (assuming suitable overloads and defaults)
var getQuote1 = new GetQuote1Task(
Timeout:TimeSpan.FromSeconds(30),
Blocking = false,
Action = (task,step) {
task.Quote1 = await SlowWebserviceCallBlockUntilReply ( ... );
});
var submitQuote2 = new SubmitQuoteTask(
Timeout:TimeSpan.FromSeconds(10),
Blocking = true,
Action = (task,step) {
task.trackingNumber = SubmitViaSlowWebServiceCall2WaitForTrackingNo( ... );
});
var waitForEmail = new WaitForQuoteByEmail(TimeSpan.FromHours(4), (task,step) {
var quote2 = readEmail(task.TrackingNo).ParseQuote2();
if ( quote2 == null ) this.Requeue(TimeSpan.FromMinutes(5), this.Elapsed);
task.Quote2 = quote2;
},
Timeout: (task,step) {
task.Quote2 = Quote.NoReply;
}
});
var checkCommission = new CheckCommissionTask(
Timeout:TimeSpan.FromSeconds(10),
Ready: task => task.Quote2!=Quote.Pending,
Blocking = true,
Action = (task,step) {
task.Quote1 = await SlowWebserviceCallBlockUntilReply ( ... );
});
var task = new GetQuoteParentTask(
Timeout :TimeSpan.FromDays(2),
Steps: getQuote1, submitQuote2, waitForEmail, checkCommission
);
queueService.Enqueue(task);
same example, all declared inline
queueService.Enqueue(
new GetQuoteParentTask(TimeSpan.FromDays(2),
new GetQuote1Task(30000, false,(task) => {
task.Quote1 = await SlowWebserviceCallBlockUntilReply ( ... );
...other code...
}),
new SubmitQuoteTask(10000,(task,step) => {
task.TrackingNumber = SubmitViaSlowWebServiceCall2WaitForTrackingNo( ... );
...other code...
}),
...waitForEmail ...,
...checkCommission..
);
if the task timeouts and code are specified in the classes, then could even be simplified as follows: TODO: need to provide class samples // define task class with nested step classes here. queueService.Enqueue(new GetQuoteParentTask( ... )); .. the GetQuote task will be elligable to pulled into play by the next worker that asks for work since it's the only enqueued task, (the child tasks steps have not been enqueud) . A seperate worker (thread) looks for tasks that have timed out and picks up those tasks and runs any timeout actions if any. Misc notes:
Please note, that the code samples above, as well as the bullet points above are not requirements, they're just ideas and notes.I'm going (as far as possible) to build this project using BDD (Behavior driven development) and start by fleshing out the requirements for a real personal project that I need long running queuing in, as .feature files in Gherkin syntax. I plan to use Raconteur to convert the feature files into Nunit tests, and will setup a build server somewhere. Cheers, Alan |