Service Graph Connector for Observability AppDynamics

[jgates]

Service Graph Connector for Observability AppDynamics

Introduction

The Service Graph Connector for Observability AppDynamics connector allows customers to ingest CMDB data from an AppDynamics installation using REST APIs and push events from AppDynamics to a ServiceNow with event management.

A high level look at the data mapping:

AppDynamics	ServiceNow
Application	Calculated Application Service (cmdb_ci_service_calculated)
Business Transaction	Calculated Application Service (cmdb_ci_service_calculated)
Tier	Calculated Application Service (cmdb_ci_service_calculated)
Node	Application (cmdb_ci_appl)
Server	Server (cmdb_ci_server)

And the service maps can be seen as follows:

From AppDynamics:

To ServiceNow:

Requirements

An active AppDynamics environment that can be reached either with or without a mid-server

The minimum AppDynamics version required is 20.3

Server Visibility will need to be active for your AppDynamics account

The following roles are necessary:

• • Applications and Dashboards Viewer (Default)
  • Server Monitoring User (Default)

For observability the Configure Http Request Templates role is required in order to create a HTTP request template from ServiceNow.

Installation

Installation of the Service Graph Connector for Observability AppDynamics can be done via the ServiceNow app store. Installation from the store will handle all necessary dependencies.

In order to get the Observability functionality, it is required to install the Observability Commons application from the ServiceNow store prior to installing the Service Graph Connector for Observability AppDynamics application from the store. The installation will check if the Observability Commons application is installed before installing the code necessary for ingesting Observability data.

Setup

Follow the Setup module under the SG Connector for Observability AppDynamics module in the left navigation.  The setup will guide the user through configuring the connection (connection and credentials, event notification setup), duplicate detection rules, advanced, and set up scheduled imports jobs.  There is documentation within the guided setup as well as articles in the right help bar during each step of this process.

Data Sources

Each data source is responsible for bringing in specific data from the AppDynamics environment into the CMDB.

• • • SG-AppDynamics Servers and Applications
    • • Contains server and application data
      • Ingests server and node data from AppDynamics
      • Targets Server (cmdb_ci_server) and Application (cmdb_ci_appl) tables in ServiceNow
      • Ingestion of nodes can be turned on and off via the sn_sg_appd.import_nodes system property (available in guided setup under Advanced)
    • SG-AppDynamics Application Services
    • • Contains application service data
      • Ingests applications, business transactions, and tier data from AppDynamics
      • Targets Calculated Application Service (cmdb_ci_service_calculated) in ServiceNow
      • Ingestion of business transactions can be turned on and off via the sn_sg_appd.import_business_transactions system property (available in guided setup under Advanced)
    • SG-AppDynamics Tier to Tier Relationship
    • • Contains relationship data between AppDynamics tiers
      • Ingests relationship between tiers within AppDynamics
      • Targets Relationships (cmdb_rel_ci) in ServiceNow
      • Turn on and off via the sn_sg_appd.populateTierToTierRelationships system property (available in guided setup under Advanced)
    • SG-AppDynamics Server Tags
    • • Contains tag information for servers
      • Ingests tag data from AppDynamics
      • Targets Key Values (cmdb_key_value) in ServiceNow
      • Turn on and off via the sn_sg_appd.populate_server_tags system property (available in guided setup under Advanced)

Service Maps

The service maps are created by the calculated application service.  The top-down flow of the service map is as follows: 

Application

Business Transaction

Tier 

Node 

Server 

The operational status is originally set to Operational but may be set to Non-Operational via the business rule Change to operational status. 

Event Management

During the guided setup of the Service Graph Connector for Observability AppDynamics there is a step to Push HTTP Request Template.  This step has a linked called Push HTTP Request Template that when clicked will execute multiple API calls to AppDynamics to setup event ingestion.  During this process the AppDynamicsAPI user will have a new secure password generated for it that AppDynamics will use to connect back to the ServiceNow instance.

This will create a HTTP Request Template in the AppDynamics environment, as seen below:

The Payload should be the following:

{
    "source": "AppDynamics"
    ,"account": {
        "entityType": "$account.entityType"
        ,"entityTypeDisplayName": "$account.entityTypeDisplayName"
        ,"id": "$account.id"
        ,"name": "$account.name"
        ,"triggerTime": "$account.triggerTime"
    }
    ,"controllerUrl": "$controllerUrl"
    ,"policy": {
        "entityType": "$policy.entityType"
        ,"entityTypeDisplayName": "$policy.entityTypeDisplayName"
        ,"id": "$policy.id"
        ,"name": "$policy.name"
        ,"digest": "$policy.digest"
        ,"digestDurationInMins": "$policy.digestDurationInMins"
    }
    ,"action": {
        "entityType": "$action.entityType"
        ,"entityTypeDisplayName": "$action.entityTypeDisplayName"
        ,"id": "$action.id"
        ,"name": "$action.name"
        ,"triggerTime": "$action.triggerTime"
    }
    ,"topSeverity": "$topSeverity"
    ,"topSeverityImage": {
        "name": "$topSeverityImage.name"
        ,"fileName": "$topSeverityImage.fileName"
        ,"mimeContentRef": "$topSeverityImage.mimeContentRef"
        ,"deepLink": "$topSeverityImage.deepLink"
    }
    ,"notificationConfigText": "$notificationConfigText"
    ,"controllerUrl": "$controllerUrl"
    ,"appDynamicsIcon": {
        "name": "$appDynamicsIcon.name"
        ,"fileName": "$appDynamicsIcon.fileName"
        ,"mimeContentRef": "$appDynamicsIcon.mimeContentRef"
        ,"deepLink": "$appDynamicsIcon.deepLink"
    }
    ,"appDynamicsLogo": {
        "name": "$appDynamicsLogo.name"
        ,"fileName": "$appDynamicsLogo.fileName"
        ,"mimeContentRef": "$appDynamicsLogo.mimeContentRef"
        ,"deepLink": "$appDynamicsLogo.deepLink"
    }
    ,"fullEventList":[
        #foreach(${event} in ${fullEventList})
        #if($foreach.index > 0)
        ,
        #end
        {
            "eventType": "$event.eventType"
            ,"id": "$event.id"
            ,"guid": "$event.guid"
            ,"eventTypeKey": "$event.eventTypeKey"
            ,"eventTime": "$event.eventTime"
            ,"displayName": "$event.displayName"
            ,"summaryMessage": "$event.summaryMessage"
            ,"eventMessage": "$event.eventMessage"
            ,"application": {
                "entityType": "$event.application.entityType"
                ,"entityTypeDisplayName": "$event.application.entityTypeDisplayName"
                ,"id": "$event.application.id"
                ,"name": "$event.application.name"
                ,"triggerTime": "$event.application.triggerTime"
            }
            ,"tier": {
                "entityType": "$event.tier.entityType"
                ,"entityTypeDisplayName": "$event.tier.entityTypeDisplayName"
                ,"id": "$event.tier.id"
                ,"name": "$event.tier.name"
                ,"triggerTime": "$event.tier.triggerTime"
            }
            ,"node": {
                "entityType": "$event.node.entityType"
                ,"entityTypeDisplayName": "$event.node.entityTypeDisplayName"
                ,"id": "$event.node.id"
                ,"name": "$event.node.name"
                ,"triggerTime": "$event.node.triggerTime"
            }
            ,"affectedEntities": [
                #foreach(${entity} in ${event.affectedEntities})
                #if($foreach.index > 0)
                ,
                #end
                {
                    "entityType": "$entity.entityType"
                    ,"entityTypeDisplayName": "$entity.entityTypeDisplayName"
                    ,"id": "$entity.id"
                    ,"name": "$entity.name"
                    ,"triggerTime": "$entity.triggerTime"
                }
                #end
            ]
            ,"healthRuleEvent": "$event.healthRuleEvent"
            #if($event.healthRuleEvent == true)
                ,"healthRule": {
                    "entityType": "$event.healthRule.entityType"
                    ,"entityTypeDisplayName": "$event.healthRule.entityTypeDisplayName"
                    ,"id": "$event.healthRule.id"
                    ,"name": "$event.healthRule.name"
                    ,"triggerTime": "$event.healthRule.triggerTime"
                }
            #end
            #if($event.healthRuleEvent == true)
                ,"incident": {
                    "entityType": "$event.incident.entityType"
                    ,"entityTypeDisplayName": "$event.incident.entityTypeDisplayName"
                    ,"id": "$event.incident.id"
                    ,"name": "$event.incident.name"
                    ,"triggerTime": "$event.incident.triggerTime"
                }
            #end
            ,"healthRuleViolationEvent": "$event.healthRuleViolationEvent"
            ,"severity": "$event.severity"
            ,"severityImage": {
                "name": "$event.severityImage.name"
                ,"fileName": "$event.severityImage.fileName"
                ,"mimeContentRef": "$event.severityImage.mimeContentRef"
                ,"deepLink": "$event.severityImage.deepLink"
            }
            ,"btPerformanceEvent": "$event.btPerformanceEvent"
            ,"deepLink": "$event.deepLink"
        }
        #end
    ]
    ,"clampLimit": "$clampLimit"
    ,"clamped": "$clamped"
}

A user can modify the HTTP Request Template in the AppDynamics application if they need further fine tuning.  Modification of the payload is not recommended as the Push Connector is defined to use the payload above.

Notifications will then be sent from AppDynamics to the EM-Connector Inbound Event API. This API will pass the information to the Push Connector (sn_em_connector_listener) defined by this integration to parse the notification and create an event if there is a corresponding CI in the CMDB. From there, standard event management deduplication and alert creation are followed.

ATF Tests

The Service Graph Connector for Observability AppDynamics integration ships with a set of OOTB ATF Tests.  These can be run by running the test suite SGO-AppDynamics Integration Tests.  These tests will put example records into the import set tables, run the transformer, and verify the output in the CMDB.  These tests can be helpful in debugging issues and verifying the environment is set up correctly in order to process the data. 

Troubleshooting

• • Validate data sources/Test load 20 records throws an error
  • • Refer to this blog for help debugging connection issues
  • Application service operational status is set to non-operational
  • • The business rule Change to operational status is responsible for setting the operational status.  It will set the operational status to non-operational if there are no entry points in the service.
  • Event connection issues between AppDynamics and ServiceNow
  • • The Push HTTP Request Template action will reset the password for the AppDynamicsAPI user.  If a HTTP request template already exists in the AppDynamics environment the new template will be rejected.  Update the AppDynamicsAPI user with a new password and update the corresponding HTTP request template in AppDynamics to reflect this new password.