Dedicated event processor

Ram117
Kilo Sage

Experts,

Today I am working on an integration with a third party application which makes some POST call to create some CI records on the other application. Since the number of records I am looking at is 100k+ , I did not want to go with the workflow / f low based approach.

I am using the script solution wherein each records will create events and script actions will make the POST API call.

Now the question I have is, I did not want these events to block the default events within the platform, hence created a dedicated event processor to process these custom events. Now while setting up the sys_tigger record, I have a question on what needs to be selected as  the 'system ID'.

  • If I do not set a value there, I see only 1 event processor getting created  
  • If I set the value as 'Active Nodes', i see the multiple trigger records getting created ( for each node 1 )

can you guide me on this  ? what is the ideal configuration which needs to be put in so as to not cause any havoc to default platform events ? 

On a side note, I am expecting my events to grow in the future. May be b/w a 1M - 2M + to be created based on the number of CI's I am going to be processing..

thx

ram.

1 ACCEPTED SOLUTION

Hello,

From my understanding and when I've used this, multiple records are created on your active nodes. What I believe the support documentation is saying is that the delegated process out of box for normal event processing, takes up to 500 events and then stops.

For your custom event, it's going to grab the entire queue...no matter how many is there, at that time.

However, your jobs on the other nodes will also do the same, and they don't run at the same exact second as each other.

So when node 1 scoops up the entire queue, once it moves on, more events are still pouring in, then node 2 goes and scoops it up and moves on, etc.

I didn't say it round robins it, I said it spreads them out. That's what I meant by my above reply.

It would be pointless to select "Active Nodes" then if the way you're thinking it works, actually worked, haha. Because if it creates 3 event processor jobs on 3 nodes, but only 1 runs...which one is it? The parent record doesn't actually run, the child jobs run, and the child jobs are on the active nodes.

Good conversation either way, feel free to submit a case to SN support and double-check with them as well, if needed 🙂

Please mark reply as Helpful/Correct, if applicable. Thanks!


Please consider marking my reply as Helpful and/or Accept Solution, if applicable. Thanks!

View solution in original post

7 REPLIES 7

Hello,

From my understanding and when I've used this, multiple records are created on your active nodes. What I believe the support documentation is saying is that the delegated process out of box for normal event processing, takes up to 500 events and then stops.

For your custom event, it's going to grab the entire queue...no matter how many is there, at that time.

However, your jobs on the other nodes will also do the same, and they don't run at the same exact second as each other.

So when node 1 scoops up the entire queue, once it moves on, more events are still pouring in, then node 2 goes and scoops it up and moves on, etc.

I didn't say it round robins it, I said it spreads them out. That's what I meant by my above reply.

It would be pointless to select "Active Nodes" then if the way you're thinking it works, actually worked, haha. Because if it creates 3 event processor jobs on 3 nodes, but only 1 runs...which one is it? The parent record doesn't actually run, the child jobs run, and the child jobs are on the active nodes.

Good conversation either way, feel free to submit a case to SN support and double-check with them as well, if needed 🙂

Please mark reply as Helpful/Correct, if applicable. Thanks!


Please consider marking my reply as Helpful and/or Accept Solution, if applicable. Thanks!

Hi,

Thanks for marking my reply as Helpful.

If it also ends up guiding you Correctly, please also mark it as Correct.

Thanks and take care! 🙂


Please consider marking my reply as Helpful and/or Accept Solution, if applicable. Thanks!

Hi Allen,

 

I have been working on a custom design for one requirement which involves firing of Multiple events everyday. I don't want to over use all the active nodes that processes the events with my events. I am looking into the possibility of using Dedicated event processor . But i have a question here. Lets say currently there are 5 nodes that are processing all the events in queue and if I'm creating a new dedicated event processor for my event, Will it take one of those 5 nodes and only use it to process all my events or will it create a new node separately just to process my events while the remaining 5 nodes still keep on processing other events.