Fine tuning event processors? Delegated vs legacy event processors

[christianmehrin]

Hi,

We have a need to process asynchronous work in logical sequence. Now all events must be processed in order, but those with same criteria. By now we have solved this by creating a customized queue having only one event processor on the instance (vs. the default queue runs one processor per node).

On high load events getting processed with a long delay. This is causes by many events in very short time. 

In that situation - and having multiple event processors - all new events would be claimed by next event processor started. The other processors would have nothing to do while the first still needs long time until processing latest event claimed.

Side effect is that newer events would be claimed by another processor and will be processed right now (while older ones already claimed still need to wait)

=> Is there a possibility to configure maximum number of events claimed on start of event processor (goal: divide load even on peeks and reduce risk of processing events not in sequence)

=> Is there a possibility to provide a callback that would return events that can be processed now?

Another idea was to set processOn to a date far away if another event with same criteria is still waiting for getting processed. When completed it should set processOn to now for next event. BR seems not to run when updating event to completed - so that would need a scheduled job. For this it would be good to add additional columns to sysevent table to directly store criteria and order (order, as resolution of time = 1s may not be sufficient)

Do you have comments on this?

regards,

Christian 

[christianmehrin]

Hi Dirk,

So far I was not able to configure other queues than default one to make use of processDelegatedEvents. Also I don't know if maximum number of event to be claimed at a time can be configured.

For my implementation I've created a new table. Instead of calling gs.eventQueue() I'm calling a customized method to populate that table. A scheduled job is checking regularly if there are new "pre-events". If so and if number of unclaimed events on sysevent for that queue is lower limit, events will be moved (up to reach this limit).

Scheduled job frequency has to be adapted based on run time of events and frequency of their event processor.

I'm using different scheduled job per queue for moving events.

regards,

Christian

[DirkRedeker]

Hi Christian

Perfect, thats how I guessed your solution will work.

Thanks a lot !! 

BR

Dirk