How Many MID Server should be recommended for a datacenter that has around 20,000 servers? and with what configuration?

[Suhas6]

Hello,

We have a scenario where there are around 20,000 servers in on-prime data center. I know that we have a MID server calculation spreadsheet which can help us to show how many MID servers will be required. 

But the first question that I have is, how many servers are recommended to be scanned by a single Instance of MID server? is there any recommended limit so as not to run into performance issues?

Also, with above 20,000 Servers, If requirement is to complete the discovery in one day for all servers, if I go for 4 MID servers (with 200 thread each) where I install MID Server 1 and MID Server 2 on One single HOST A and MID Server 3 and MID Server 4 on Host B and then distribute the IP ranges for discovery among these MID servers to get discovery completed throughout night Will this architecture work? or is there any better way to do this? Also, Since there will be other environments like TEST, DEV etc.. I will also need may be similar structure but with less MID server instances.

Please recommend asap.

Thank you,

Suhas

[tim_broberg]

So, with 2 mids and 25 threads, I see 40k probes run in about an hour at ~20 probes per server ~= 2k servers?

You have 2x the mids with 8x the threads = 16x the power to do 10x the servers, so one would think this would take about an hour.

What I wonder is, what's going to happen to the worker threads on the instance when you try to run 500k sensors in an hour? How many nodes do you have? 8 workers per node?

I'm seeing an average sensor duration of about 400worker-ms / sensor x 500k sensors / hour = 200kworker-S / hour. Your mileage will most definitely vary, depending on what kind of devices you're discovering and how.

200kworker-S / hour x 1 hour / 3600 S = 56 workers / 8 workers / node = 7 nodes.

So, you could keep an 8-node instance 7/8 busy for an hour discovering at this pace.

That's a lot of load.

You could try it, but I'm thinking you might want to decrease the thread count to 100 or 50 or cut back on the mid count to stretch out the duration of the discovery to 2 or 4 hours so as to reduce the load on the worker threads.

(I understand that the worker scheduling is improving in New York. We'll see what that means as far as responsiveness during sensor storms.)

Hope this helps, and I'm sure we would be interested to hear what you learn when you try it out,
    - Tim.