Event-Based Cloud Discovery in ServiceNow: Real Time AWS Infrastructure Visibility

Vishnu-K · 4 hours ago

Article

The problem with scheduled cloud discovery is obvious once you think about it: you're always behind.

You run discovery on a schedule—daily, maybe weekly—and your CMDB is accurate... until something changes.

EC2 instances spin up and tear down. RDS databases get modified. Load balancers get security group rules changed. Tags get updated. And your CMDB doesn't know about any of it until the next scheduled discovery run.

That's where event-based cloud discovery changes the game.

Instead of polling AWS on a schedule, ServiceNow can now react instantly when AWS detects a change. The moment an EC2 instance launches, an RDS database configuration changes, or security groups get updated, AWS fires an event. ServiceNow catches that event and automatically triggers targeted discovery for just that resource. Your CMDB updates in near real-time.

This isn't a minor convenience. It's fundamental to how modern operations should work.

How Event-Based Discovery Actually Works

The architecture is elegant in its simplicity:

AWS detects a change using AWS Config—a service that continuously monitors your AWS resources and tracks every configuration change. Whether it's a state change (instance starting/stopping), a tag modification, or a security group update, AWS Config sees it.

AWS fires an event to an SNS (Simple Notification Service) topic. This happens immediately when the change is detected, not on any schedule.

ServiceNow receives the event through an SNS subscription. The event lands in ServiceNow's Cloud Events table as "Ready" for processing.

The Cloud Event Scheduler processes the event by triggering either response mappings (the fast path) or targeted discovery patterns (the complete path). Response mappings are lightweight field mappings that quickly update the CMDB with core attributes. If response mappings aren't available, ServiceNow runs the appropriate discovery pattern to gather full details about the affected resource.

Your CMDB updates automatically with the latest information about that specific resource—its configuration, tags, relationships, everything.

The entire flow happens in seconds, not hours.

The AWS Side: Setting Up AWS Config and SNS

This is where most implementations stumble because there are multiple approaches, and documentation doesn't always clarify which one to use.

Step 1: Enable AWS Config

AWS Config is your change detection engine. It continuously evaluates your AWS resources against rules you define and tracks all configuration changes in a central recorder.

In the AWS console:

Navigate to AWS Config
Select "Get Started"
Choose the resources you want to monitor (usually "all resources")
Configure an S3 bucket for the delivery channel (AWS Config stores configuration history here)
Assign an IAM role with appropriate permissions

Step 2: Create SNS Topic for Event Distribution

This is where AWS sends notifications when changes occur.

In AWS SNS console, create a new topic (name it something like servicenow-cloud-discovery-events)
Keep it standard (not FIFO unless you have specific ordering requirements)
Configure the access policy to allow AWS Config to publish messages

Step 3: Configure AWS Config Delivery

Tell AWS Config to send notifications to your SNS topic when changes are detected:

In AWS Config settings, enable notifications
Link to your SNS topic
Configure what types of events trigger notifications (configuration item changes, compliance changes, etc.)

Step 4: Grant ServiceNow Access to Receive Events

Create an SNS subscription that endpoints to ServiceNow:

In SNS console, create a subscription to your topic
Set protocol to "HTTPS"
Set the endpoint to your ServiceNow instance's inbound email address or API endpoint (the exact URL depends on your ServiceNow setup)

ServiceNow will receive a SubscriptionConfirmation event first—you need to confirm this subscription in ServiceNow before events start flowing.

The ServiceNow Side: Configuration and Activation

Now configure ServiceNow to process those incoming events.

Step 1: Install Cloud Provisioning & Governance Plugin

Event-based discovery requires the Cloud Provisioning and Governance module. It includes:

Cloud Config Management
Cloud API
Event processing engines

These plugins include response mappings and targeted discovery patterns out of the box.

Step 2: Configure Cloud Service Accounts

Set up your AWS accounts in ServiceNow (master and member accounts) as you would for standard cloud discovery:

Navigate to Cloud Provisioning > Setup > Cloud Service Accounts
Create records for each AWS account
Mark which is the master account (the one with overall discovery authority)
Configure cross-assume role parameters to enable multi-account access

Step 3: Enable and Configure Cloud Event Scheduler

This is the component that processes incoming events:

Go to Cloud Provisioning > Setup > Cloud Event Scheduler
Configure the number of parallel schedulers (this affects event processing throughput)
Set property sn_cmp.cloud_event.use_response_mapping_aws to True (default in Zurich+)
When this is True, ServiceNow uses response mappings first (fast), then falls back to patterns if needed
Optionally set sn_cmp.cloud_event.parallel_scheduler_count to scale event processing

Step 4: Configure Event Subscriptions

Tell ServiceNow which AWS service account's events to listen for:

Go to Cloud Provisioning > Events > Cloud Event Subscriptions
Create a subscription for your AWS service account
Enable the subscription
Choose which event types to process (most commonly: ConfigurationItemChangeNotification)

Step 5: Establish SNS Subscription Confirmation

When AWS sends the SubscriptionConfirmation event, ServiceNow needs to confirm it:

The event will land in your Cloud Events table
ServiceNow automatically detects this subscription confirmation event
The subscription is confirmed, and regular events can now flow

The Real-Time Discovery Workflow

Once configured, here's what actually happens:

You create a new EC2 instance in AWS. Within seconds:

AWS Config detects the new resource and evaluates its configuration
AWS Config publishes to SNS with details about the new instance
SNS delivers to ServiceNow with a ConfigurationItemChangeNotification message
ServiceNow Cloud Event Scheduler picks up the event and marks it as "Ready"
Event processing begins: If a response mapping exists for EC2 instances, it updates the CMDB directly. Otherwise, it triggers the EC2 discovery pattern.
The pattern gathers complete details: instance ID, instance type, AMI ID, security groups, subnet, tags, state, etc.
A new CI is created in your CMDB (or an existing one is updated)
Relationships are updated: links to security groups, subnets, VPCs, accounts

All of this happens automatically. No manual intervention. No waiting for a scheduled discovery run.

Advanced Configuration: Response Mappings

Response mappings are where this gets powerful. Instead of running full discovery patterns every time (which is resource-intensive), ServiceNow uses lightweight mappings to extract data directly from the AWS Config event and push it straight into the CMDB.

For example, an EC2 response mapping might capture:

Instance state (running, stopped, terminated)
Instance type
AMI ID
Tags
Security groups
VPC and subnet information

All from the raw event data without needing to query AWS again.

You can create custom response mappings if ServiceNow's defaults don't cover your specific needs. They're defined in the Cloud Provisioning > Event Processing > Response Mappings module.

Monitoring Event Processing

Because event-based discovery is automated, you need visibility into what's happening:

Cloud Events Table (sn_cmp_cloud_event): Shows all incoming events with their status

Ready: Waiting to be processed
Processing: Currently being handled by the scheduler
Processed: Successfully completed
Failed: Something went wrong

Cloud Event Logs: Detailed execution logs showing what actions were taken for each event

Cloud Event Scheduler Logs: Execution history of the scheduler itself

If events start failing, check:

Are response mappings available for this resource type?
Are discovery patterns available as a fallback?
Do your IAM roles have permission to query the resource?
Is the Cloud Event Scheduler active and running?

Best Practices for Event-Based Discovery

Start with scheduled discovery plus events: Don't immediately go event-only. Run scheduled discovery daily (or weekly) as a baseline to catch any events that might have been missed, plus full synchronization. Add event-based discovery on top for the incremental changes.

Filter by region: If you're only using specific AWS regions, configure AWS Config to only monitor those regions. This reduces event noise and improves performance.

Use tags strategically: Tags on AWS resources drive a lot of automation. If an EC2 instance is tagged with servicenow-exclude: true, you can configure discovery to skip it. Consistent tagging means better discovery.

Monitor parallel scheduler count: If you have thousands of AWS resources and frequent changes, you'll get a high volume of events. Scale your parallel schedulers accordingly. Start conservative and increase based on queue length.

Set a default domain: If event processing fails to identify a service account domain, set the sn_cmp.error_events.default_domain property. This prevents failed events from appearing to all domains and polluting CMDB data.

Test the SNS endpoint first: Before wiring everything together, test that SNS can actually reach your ServiceNow HTTPS endpoint. Network connectivity, SSL certificates, and firewall rules are common failure points.

Common Challenges and Solutions

Events aren't showing up: Check that AWS Config is enabled and publishing to SNS. Verify the SNS subscription is confirmed in ServiceNow. Check network connectivity between AWS and your ServiceNow instance.

Events are stuck in "Ready" status: The Cloud Event Scheduler might not be running. Navigate to Cloud Provisioning > Setup > Cloud Event Scheduler and verify the scheduler is active. Check system logs for errors.

Processing is slow: You might need more parallel schedulers. If the event backlog is growing, increase sn_cmp.cloud_event.parallel_scheduler_count. Also check if patterns are running when response mappings should be sufficient.

Resource isn't in CMDB after event: A response mapping failed and the pattern didn't run. Check if the pattern exists for that resource type. Verify IAM permissions allow discovery of that specific resource.

Why This Matters for Your Operations

Event-based discovery solves a fundamental problem: your CMDB is always stale unless you're actively keeping it updated.

With traditional scheduled discovery, you get snapshots. With event-based discovery, you get continuous synchronization. Your CMDB becomes a live representation of your actual infrastructure, not yesterday's version.

This changes everything downstream:

Incident management: When an incident is created, you query the CMDB and get current infrastructure state, not stale data
Change management: You can see what actually changed and what dependencies exist right now
Service mapping: Dynamic dependency maps based on current infrastructure state
Cost management: Real-time tagging changes mean cost allocation updates immediately
Compliance: Configuration drift is detected and recorded instantly

The infrastructure world has moved to event-driven automation. Event-based cloud discovery brings your CMDB into that world.

Next Steps

If you're currently running scheduled cloud discovery, adding event-based discovery is a natural next step:

Ensure Cloud Provisioning & Governance is installed
Enable AWS Config and create the SNS topic (if not already done)
Configure the SNS subscription to ServiceNow
Enable the Cloud Event Scheduler in ServiceNow
Start with a low parallel scheduler count and monitor
As confidence builds, reduce your scheduled discovery frequency (keeping it as a weekly full sync for safety)

The first time you see your CMDB update within seconds of making an AWS infrastructure change, you'll understand why this matters. It's the difference between managing infrastructure reactively and proactively.

"Happy Learning 😊"

Please mark this as helpful if you found it useful.