Secure by Design Connectivity

shazza72
ServiceNow Employee

Sunday

Executive Summary

Public-sector organizations face unique challenges when adopting ServiceNow’s Protected Platform (SPP), particularly around stringent security and networking requirements. These constraints complicate discovery, integration, and operational resilience, especially for agencies handling PROTECTED data. This whitepaper presents a case study from a major Australian public-sector utility, detailing the architecture decisions, trade-offs, and governance model that enabled a secure, compliant, and resilient ServiceNow deployment.

The document covers ExpressRoute considerations, Netskope proxy configuration, MID server clustering, API Gateway mediation, and pre-production security testing. It also integrates lessons learned and stakeholder feedback, providing actionable guidance for agencies seeking similar outcomes.

Background: Why Connectivity Matters

ServiceNow’s SPP architecture leverages MID servers to bridge cloud and on-premises environments. While this model simplifies integration, it introduces complex design questions around routing, failover, and compliance. For the public-sector utility, these questions were amplified by government mandates for data sovereignty, penetration testing, and tightly controlled ingress/egress paths.

Objectives:

  • Secure by design connectivity for all ServiceNow traffic
  • Operational resilience through failover and monitoring
  • Compliance alignment with government security frameworks

Connectivity Model Overview

Key Clarification Points for SPP AU Connectivity

ServiceNow SPP AU instances are assigned public IP addresses, but these are IP-restricted by default. Customers must provide ServiceNow with a range of source IP addresses to whitelist, regardless of whether connections are established over ExpressRoute or the public Internet. This ensures that only authorised traffic can access the SPP AU instance, enhancing security posture while maintaining operational flexibility.

Each ServiceNow instance comprises two or more application nodes and a dedicated database node. All applications, portals, and web service endpoints are accessed via the same public IP, simplifying firewall and access management. This unified interface covers both ServiceNow end-user access and on-premises MID server connectivity.

Connectivity and Security Architecture

  • Public IP Interface: ServiceNow provides a common public IP-based interface for all inbound SPP connectivity, including both end-user and MID server access.
  • MID Server Proxy Support: The MID Server supports forward proxy configurations, including compatibility with solutions like Palo Alto, enabling organizations to enforce outbound security policies.
  • Azure Backbone Considerations: Any connectivity segment traversing the Azure backbone should be treated as public Internet, as it is external to the ServiceNow network. This means that PROTECTED data in transit from the MID Server to SPP is exposed to the public Internet, even if ExpressRoute is used for part of the path.
  • Internet Dependency: If the public-sector utility disconnects from the Internet such as during a malicious attack; SPP connectivity from the utility’s data center will also be lost, since the connection is not a fully private link. This holds true regardless of ExpressRoute usage.
  • NetSkope Proxy Path: The current pattern for end-user access to SPP via NetSkope proxy over the public Internet involves PROTECTED information
  • Outbound Traffic: All outbound SPP connections to third-party ServiceNow commercial cloud instances are Internet based over HTTPS.
  • MS Teams/Entra ID Integration: Connectivity between SPP and Microsoft Teams/Entra ID occurs over the Azure backbone, as confirmed with Microsoft.
  • Preferred Routing: In principle, ExpressRoute is preferred over the NetSkope path for PROTECTED data transmission.
  • High Availability: Failover between SPP high availability instances is transparent to the utility; all connections are managed via a single load balancer.

High Level Architecture

The public-sector utility’s ServiceNow SPP deployment is anchored by a robust, multi-layered connectivity model designed to balance security, compliance, and operational agility. The architecture integrates several core components and patterns, each serving a distinct role in ensuring secure and resilient service delivery.

shazza72_0-1768194364786.png

 

Core Components

  1. ServiceNow ECC Queue:
    At the heart of the integration is ServiceNow’s External Communication Channel (ECC) queue. This mechanism orchestrates communication between the ServiceNow instance and MID servers. The instance never initiates outbound connections; instead, MID servers poll the ECC queue for work, ensuring that all traffic is initiated from within the trusted network perimeter.
  2. MID Servers:
    MID servers are strategically positioned within secure network zones, acting as the bridge between on-premises systems and the ServiceNow cloud. These servers are clustered for high availability and configured to support forward proxy solutions (such as Palo Alto), enabling granular control over outbound traffic and facilitating compliance with internal security policies.
  3. ExpressRoute and Netskope Proxy:
    The architecture supports two primary connectivity patterns for inbound traffic:
  • ExpressRoute: Preferred for PROTECTED data, ExpressRoute provides a dedicated, private link between the utility’s network and Microsoft Azure, reducing exposure to the public Internet and enhancing security for sensitive workflows.
  • Netskope Proxy: For browser and API ingress flows, the utility has implemented Netskope proxy configurations. This pattern is used for scenarios where ExpressRoute is not feasible or where additional inspection and policy enforcement are required.
  1. API Gateway Mediation:
    API Gateways are deployed to mediate and secure API traffic between the utility’s systems and ServiceNow. These gateways enforce authentication, authorisation, and traffic shaping policies, providing an additional layer of defence against unauthorised access and potential threats.
  2. Public IP Interface and IP Whitelisting:
    ServiceNow SPP AU instances are assigned public IP addresses, but access is strictly IP restricted. The utility provides ServiceNow with a range of source IPs to whitelist, ensuring that only authorised connections whether via ExpressRoute or public Internet are permitted. All applications, portals, and web service endpoints are accessed over this single public IP, simplifying firewall management and reducing the attack surface.
  3. Azure Backbone Considerations:
    Any connectivity segment traversing the Azure backbone is treated as public Internet, as it is external to the ServiceNow network. This means that PROTECTED data in transit from the MID Server to SPP is exposed to the public Internet, even if ExpressRoute is used for part of the path. The architecture accounts for this by implementing encryption and monitoring at all critical junctures.
  4. High Availability and Load Balancing:
    Failover between SPP high-availability instances is transparent to the utility. All connections are managed via a single load balancer, ensuring seamless continuity of service in the event of node or network failures.

Traffic Flow and Security Controls

  • Inbound Traffic:
    End user and MID server connections to SPP are routed through either ExpressRoute or Netskope proxy, depending on the sensitivity of the data and operational requirements. All inbound traffic is subject to IP whitelisting and firewall rules.
  • Outbound Traffic:
    All outbound connections from SPP to third-party ServiceNow commercial cloud instances are Internet-based over HTTPS, with strict controls on allowed destinations and protocols.
  • Integration with Microsoft Services:
    Connectivity between SPP and Microsoft Teams/Entra ID occurs over the Azure backbone, leveraging existing enterprise authentication and collaboration frameworks.

Monitoring and Logging

Robust monitoring and logging are foundational to the security, compliance, and operational resilience of the public-sector utility’s ServiceNow SPP deployment. The architecture incorporates multiple layers of visibility and control, ensuring that all connectivity segments are continuously observed, and that actionable intelligence is available for both proactive management and incident response.

MID Server Monitoring and Logging

  • Activity Logging:
    All MID servers are configured to log operational activity, including job execution, authentication attempts, configuration changes, and system events. Logs are securely forwarded to a centralised Security Information and Event Management (SIEM) platform for aggregation and analysis.
  • Health and Availability Monitoring:
    Automated health checks monitor MID server status, resource utilization (CPU, memory, disk), and network connectivity. Alerts are generated for anomalies such as service failures, high resource consumption, or loss of connectivity to the ServiceNow instance.
  • Audit Trails:
    Detailed audit trails are maintained for all administrative actions on MID servers, supporting compliance with government frameworks and enabling forensic investigations if required.

Network and Connectivity Monitoring

  • Traffic Flow Analysis:
    All inbound and outbound traffic between the utility’s network, ExpressRoute, Netskope proxy, and ServiceNow SPP is monitored for volume, protocol, and destination. Flow logs are retained for trend analysis and anomaly detection.
  • Firewall and Proxy Logging:
    Firewall and proxy devices log all connection attempts, both allowed and denied. These logs are correlated with MID server and application logs to provide end-to-end visibility of user and system activity.
  • Encryption and Integrity Checks:
    Network monitoring tools verify that all sensitive data in transit is encrypted (e.g., TLS/SSL) and that no unauthorised protocol downgrades or certificate anomalies occur.

Application and API Monitoring

  • API Gateway Logging:
    All API requests and responses passing through the API Gateway are logged, including metadata such as source, destination, method, and response code. Rate limiting and anomaly detection rules are enforced to identify potential abuse or misconfiguration.
  • ServiceNow Instance Logging:
    The ServiceNow platform’s native logging capabilities are leveraged to capture application-level events, integration activity, and user access patterns.

Security Event Management

  • Centralized SIEM Integration:
    All logs from MID servers, network devices, proxies, and ServiceNow are ingested into a centralised SIEM. This enables:
    • Real-time correlation of events across the environment
    • Automated alerting for suspicious or policy-violating activity
    • Dashboards for compliance reporting and operational oversight
  • Incident Response Playbooks:
    Predefined playbooks guide the response to detected incidents, such as unauthorised access attempts, failed jobs, or unexpected data flows. These playbooks are regularly tested and updated.

Compliance and Retention

  • Log Retention Policies:
    Log data is retained in accordance with government and organizational policies, typically for a minimum of 12 months, to support audits and investigations.
  • Access Controls:
    Access to logs and monitoring dashboards is strictly controlled and audited, ensuring that only authorised personnel can view or manage sensitive monitoring data.

Continuous Improvement

  • Regular Review:
    Monitoring and logging configurations are reviewed quarterly by the architecture and security teams to ensure coverage remains aligned with evolving threats and compliance requirements.
  • Lessons Learned:
    Insights from incidents, near-misses, and routine operations are used to refine monitoring rules, alert thresholds, and response procedures.

 

Security Testing & Validation

Pre-production penetration testing was a cornerstone of the approach. The scope included MID servers, API endpoints, and ExpressRoute routing paths. Findings were remediated before go-live, reducing residual risk and satisfying compliance auditors. The utility’s insistence on conducting its own penetration tests, rather than relying solely on vendor certifications, was critical for internal buy-in and risk management.

Implementation Checklist

  • Connectivity prerequisites: Validate firewall rules, proxy chains, and ExpressRoute circuits.
  • MID server hardening: Apply OS security baselines, rotate credentials, enable logging.
  • Cluster configuration: Configure MID clusters and test failover scenarios.
  • Pen-test readiness: Document endpoints, prepare test data, align with security teams.

 

Actions and Way Forward

To address these architectural and governance considerations, the following actions are recommended:

  1. Proof of Concept (POC):
    Set up a POC for on-premises MID Server connectivity to SPP via ExpressRoute, utilizing Palo Alto as a forward proxy. This will test proxy compatibility for the MID Server to SPP connection.
    Action: Public-sector utility to lead.
  2. Risk and Governance Review:
    Internally review and agree on the implications of:
    • Public Internet exposure of PROTECTED data between the utility and SPP.
    • Potential unavailability of SPP if the utility disconnects from the Internet. These points must be formally addressed with program governance/PSG.
      Action: Public-sector utility to lead.
  3. Connectivity Pattern Assessment:
    Conduct an internal assessment of both ExpressRoute and NetSkope options for MID Server and end-user access, based on the above clarifications. Formally agree on a preferred connectivity pattern and document the rationale.
    Action: Public-sector utility to lead.

Governance Snapshot

Roles and decision rights were formalised across dual environments, supported by architecture boards and cadence based reviews. A RACI matrix was established for connectivity decisions, and governance artifacts such as meeting minutes and decision logs were maintained to ensure transparency and accountability.

Outcomes & Lessons Learned

  • Reduced integration risk through clear routing and clustering strategies.
  • Improved resilience with automated failover for API driven workflows.
  • Compliance confidence via early pen-testing and documented governance.

Conclusion

Secure connectivity is not a one size fits all exercise. This case demonstrates that combining Netskope, MID server clustering, and API Gateway mediation underpinned by rigorous governance can deliver a compliant, resilient ServiceNow deployment for public-sector agencies. The experience of the public-sector utility highlights the value of proactive risk management, stakeholder engagement, and continuous improvement.

Next Steps for Readers

  • Assess your current connectivity model against the patterns outlined here. Refer to the SPP documentation provided by the ServiceNow CISO.
  • Implement decision matrices for integration types and failover strategies.
  • Schedule pre-production penetration testing early to validate posture.
  • Establish clear governance structures and maintain transparency through documentation.

Core SPP Links & Documentation

  1. MID Server and Connectivity Requirements
  1. Load Testing and Performance
  1. Penetration Testing and Security
  1. Disaster Recovery and High Availability
  1. Integration Patterns and APIs
  1. Security Best Practices and Instance Hardening
  1. Connectivity Testing
  • SPP Connectivity Test Host:
    https://hitide.servicenowcloud.com.au
    For verifying network connectivity and failover/backup links to SPP.
    • Current IP addresses for HITIDE:
      • 20.53.219.2
      • 20.53.10.2

 

For further information or to discuss how these patterns can be adapted to your organization, please contact your ServiceNow representative or the architecture team at your agency.

 

Labels:
0 Helpfuls
  • 225 Views
Version history
Last update:
Sunday
Updated by:
ServiceNow Employee
Contributors