Out of the box ServiceNow platform does a great job of collecting the important data from cloud platforms, as you can see from the breakdown of data we collect from AWS.
However, new properties are always being added to CMPs, and sometimes we need just need a specific lesser used property to be collected to enrich our CMDB with the data needed to generate a certain report or drive some orchestration use case.
Fortunately, the ServiceNow platform is extensible and dynamic, so it's easy to extend the out-of-the-box properties to discover that property you need!
For AWS two tables are used to specify how Discovery will process properties, the first is the AWS Datasource type mapping table, this tells discovery how to process the properties returned and map them to an object, and the second is the Normalized Object to DB Mapping table, which tells the system how to map that object to a specific field in the CMDB.
Lets cover a specific example of using these tables to collect an additional property; I want to be able to tell at a glance when my EBS volumes were attached to my instances, and run reports on this based on data in the CMDB. In general, this can also give a good indication of Instance creation time (even if the instance was not created from the ServiceNow platform) as more often than not the root device is attached on creation and never changed (and AWS does not store instance creation time), but the platform doesn't collect this data out of the box….so lets expand the collection to add it to our AWS Discovery process.
First we need to create a field to store our new property. Type Table in the search box and select System Definition > Tables. On the tables page, search for "cmdb_ci_aws_ebs_volume"
Open the table definition by clicking the , and then click "New" to create a new table column.
Then we define our new volume Attach Time field on the EBS Volumes table with the following properties.
Now we have our table configured with our new property we need to add our datasource mappings to map the data we want discovery to capture from the AWS payload and store in the field we just created.
In the tables page, search for "aws_datasource_type_mapping" and open the table by clicking the .
Scroll down and select "Show List" from the related links section.
Now from the list search for the Normalized Property "blockDeviceMapping". Select the entry for resource type Instance and open this entry using the for the row.
This particular mapping uses a script to build an object representing the volume and is used to process the block devices returned to us from AWS, we will modify it to add processing for the attach time of the EBS volumes.
Add the following line to the script after line 20
vol.attachTime = items[i].ebs.attachTime;
I also added a comment above to highlight the change but this is optional. Click "Update" to save your changes.
Now that we have expanded the payload processing to include the Attach Time information, we need to map this to our Attach Time field, this is done using the normalized mapping table.
Search for "normalizedobject_db_mapping" in the Tables view and open the table using the .
Scroll down to the Related Links section and select "Show List" again, then search for the Normalized Property blockDeviceMapping and click the blue to open the record.
The EBS volume properties are stored in an object, so mapping a new volume property is a two step process. First, we need to update the attachmentSet object to include our new Attach Time value.
Edit the script and add the following after line 22
"attachTime": volumes[i].attachTime,
so that it looks like the following (again the comment is optional).
Click update to save your changes.
Now that the EBS attachmentSet contains our Attach Time property, we need to map that property to the field we created on the EBS volumes table.
Go back to the Normalized Object to DB Mapping table and click "Show List" under related links, from this view click "New" to create a new entry.
In the form that opens, fill out the new record as per below.
Check the "Use script box" and replace everything below line 13 with the following script.
function getOutputData(params) {
var s = params.normalizedObject.attachmentSet.attachTime;
var attachTime = '';
// Process the date/time into an SN acceptable format
if (s !== undefined) {
var bits = s.split(/[-TZ:+\.]/g);
attachTime += bits[0];
attachTime += "-";
attachTime += bits[1];
attachTime += "-";
attachTime += bits[2];
attachTime += " ";
attachTime += bits[3];
attachTime += ":";
attachTime += bits[4];
attachTime += ":";
attachTime += bits[5];
}
return attachTime;
}
And we're done! After the next AWS discovery runs each entry in the EBS Volumes table will start having the Attach Time field populated, as long as the attachment time exists in AWS (obviously the volume needs to be attached to something!).
If you want to kick off a discovery immediately to check things are working, you can kick off a manual discovery run by clicking "Discover now" under AWS Discover > Discovery Schedules > Your AWS account.
Once we have this data stored in the CMDB we can make some tweaks to the views make it show up in useful places.
For example, to make the property show in the list view of EBS Volumes, browse to the EBS Volumes page and click the gear icon.
This will present you with the option to personalize the columns you see in the list view. In this case I have selected our new Attach Time property and clicked the right arrow to move it into the Selected properties list, then using the up arrow I have made the property appear directly after volume ID.
This gives us a view that looks like this:
You can also view this information in the EC2 instances views directly, which is handy, as we originally mentioned, to get an idea of when a VM was created as AWS does not store creation time for an instance, only last start/launch time, but quite often creation time is the same as the attach time of the root block device (unless you've replaced the root device without recreating the instance for some reason).
When viewing an EC2 instance details, right click the header and select Configure > Form Layout
This will bring you up a list of properties again. Scroll down the list of available properties and select "AWS EBS Volume->Instance" then use the right arrive to add it to the selected properties list.
You will now see details of the EBS volumes related to the Instance directly on the EC2 instance form, including our new Attach Time property.
NOTE: if you don't see Attach Time in the list, click the gear icon and add it to the selected properties list again as we did for the EBS volumes list view, this will save the selected properties for this particular view context so you should then see it on every EC2 instance form.
You can expand your collection to collect any property returned by the API of AWS/Azure in this way, it's simply a matter of creating a field to store it and then adding the property to the mappings tables, and if it's a single property not a subset property no scripting is required. Hope this was useful, thanks for reading!
