Experiences Implementing ServiceNow Discovery (Part 1 of 2)

[sabell2012]

A few days ago we had an outside company contact our group with questions on what it took us (resources, time, etc.) to implement ServiceNow Discovery in our environment. Since we have been running Discovery in our environment since March 2012 they were hoping to get a feel for what it would involve to implement Discovery at their company, and were also interested in any problems or pitfalls they should be aware of. What follows is a summary of my response (well, with some areas expanded for further clarification, and others added that I missed, and…so I guess it qualifies as more than a summary…).

Do not get me wrong. Discovery is awesome! We will never go back. It rocks! The data is spectacular! Asset Management would hurt us if we turned it off! You get the idea. You just have to be careful in the implementation and know the up-front costs.

Part 1 will describe implementation and sizing observations. Part 2 will describe things to watch out for, and recommended best practices with implementation.


Let me start by stating that:

[*]Discovery is an add-on product sold separately by ServiceNow. This will need to be purchased. Make sure you discuss and understand Discovery costs with your ServiceNow account representative. There are a couple of different models, and you need to be aware of how ServiceNow charges for Discovery. If your operation is small, or you only plan to Discover a small subset of devices then I suggest looking at Help-the-Help-Desk. We implemented both to allow for the fullest coverage of off and onsite devices.
[*]Discovery is not Asset Management! This is a common misconception that I have run into (in our own company).
[*]Discovery does not take Asset Management's place. Amazingly this is another misconception I keep running into with management! That or some sort of belief that our team is either part of OR actually is the Asset Management team! Our team provides and supports the tools that Asset Management uses dad-gummit! (at this point I am jumping up and down, turning red in the face, and frothing at the mouth … breathe in … breathe out … breathe in ...).
[*]Discovery instead supports and confirms Asset Management, and automatically fills in many of the blanks with discovered devices. It is a labor saving mechanism.
[*]Discovery can only discover what is on the network AND what actually responds to the queries. It cannot discover monitors (seriously), keyboards, mice, … you get the idea! There are interesting problems with discovering some phone systems (Avaya for one), ip cameras, etc. that will have to be worked through by your discovery team.
[*]Segue: Yes, discovery will be a team effort. You will need not only you the ServiceNow Admin/Developer, but someone (preferably senior) in Asset Management, and someone (preferably senior) in System/Network engineering.
[*]A CMDB that is in pretty good shape; data and structure-wise. Ours was a mess. The out-of-the-box (OOtB) structures had been modified by my predecessors to such an extent that it took our team awhile to straighten that out before we could proceed with turning Discovery on.


Some realistic observations:

[*]Implement the Asset Management plug-in. This will require training the Asset Management (AM) team to use. It splits out most AM fields from CMDB_CI. More on the split later.
[list=a]
[*]Move to Calgary Patch 2 (or greater) soonest to get the most functionality out of Discovery and AM. More on patch 2 later.


Sizing:

[*]You will need to configure at least one MIDServer box.

[list=a]
[*]This can be a Virtual Machine (VM), but I recommend that the box be dedicated entirely to MIDServer use. This will allow your network engineers to tailor the firewalls, routers, etc. for that specific box (make sure you have a dedicated IP as well!). See the MIDServer wiki(s) for details on hardware and port requirements. My recommendation is: Windows Server 2008R2 or 2012, 2 Gb RAM, 40 Gb HD, 2 processors (medium power). This will allow you to configure 2 -3 MIDServer services on your boxes in parallel. Something nice for doing DEV and QA at a later time. I do NOT recommend running two production services in parallel. See the ServiceNow wiki on implementing two MIDServers on the same box.
[*]I was bumping into issues with Calgary Patch 1 on Windows Server 2012 (HI ticket currently being worked on this). This was resolved. Turned out to be missing communications rules on our side. Suspect this if you have authentication issues, and nothing appears to be incorrect with the configs!
[*]For speed it is better to have one MIDServer for each subnet. I recommend breaking yours out by site. MIDServer services are free. You may install as many as you need. Just remember that as the number goes up so do the maintenance costs. You can run the jobs for each subnet one-after-the other. We balance our jobs, and some servers only perform one job per day.
[*]You will need to work with adjusting your discovery jobs. I am constantly playing around to get the best balance. With some I use the b-level (Example: 192.168.*.*/16). With others the c-level. With the c-level you can end up maintaining a LOT of ranges after a very short period of time. Warning: from the b-level will cause the job to take quite a bit of time as it has to check 65k+ ip addresses. Depending on the number of devices it could conceivably cause the job to run between 2-4 hours. Smaller is faster, but costs go up in maintenance. You will spend some time fiddling with the schedules on these (especially if you only have a couple of MIDServers).


[*]Implementation will require at least one full-time person. Once implemented this will continue to absorb most of one person for several months (depending on your scope). We have several thousand desktops, laptops, servers, network devices, etc. that were discovered. You will have anomalies that will need personnel to focus on resolutions. Expect to work with your AM team a lot!

Recommended personnel for implementation:

[list=a]
[*]1 ServiceNow admin (preferably senior)
[*]1 ServiceNow admin (junior — in training; and for backup so your senior admin can go on several very well deserved vacations)
[*]1 Quality Assurance head (for testing)
[*]0.5 Business Analyst (during planning)
[*]0.5 Project Manager (for project life cycle)
[*]Time from Network Engineering (network device configs)
[*]Time from System Engineering (server, desktop configs)
[*]Time from AM team (1 to 2 heads, 2-3hrs / day for a few weeks depending on scope).
[*]POST implementation: 0.5 to 1 ServiceNow Admin. Eventually you could conceivably reduce this down to 0.25 Admin as mystery devices are resolved. This would change upward, of course, when implementing new releases of SeviceNow. Additionally I recommend a designated contact in the AM team to work resolutions with. All post implementation modifications are considered change requests


[*]Time for completion for our team: one month from start to roll-out. Roll-out was delayed due to the need for verifying the data being brought in, and rectifying devices that were initially not responding. After implementation it is not uncommon to have a considerable number of network devices that are still not responding to SNMP (or SSH) calls. This is normal and expected. Most will be device configuration issues. There are a lot of non-discoverable items (IP phones, IP cameras). These come back as Active, but since there is not enough information retrieved they never classify.

[list=a]
[*]Discovery is a process that is for helping Asset Management verify and validate hardware, virtual machines, and software. It is NOT a replacement to Asset Management (Seems I have mentioned this before…!).


End of Part 1.

Steven Bell
Santander Consumer USA, inc.
Dallas, TX