Domain separation and Stream Connect

  • Release version: Yokohama
  • Updated January 30, 2025
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Domain separation and Stream Connect

    Domain separation in Stream Connect enables ServiceNow customers to logically segregate data, processes, and administrative tasks into distinct domains. This separation controls user access and visibility, ensuring that data and processes remain isolated per tenant or business unit within a single ServiceNow instance. Stream Connect supports domain separation as a standard feature, allowing service providers to manage multiple customers (tenants) securely and efficiently.

    Show full answer Show less

    Key Features

    • Message Replication: Tables related to message replication such as Message Replications, Channel Replications, Message Replication Statistics, and Kafka Topic Replications are domain separated. Domains are assigned based on reference fields, ensuring that replication data remains isolated per domain.
    • Topic Namespaces: Kafka topics are linked to namespaces, which in turn are linked to ServiceNow domains. This linkage restricts topic access so that users can only view and interact with topics assigned to their domains, controlled further by domain visibility and ACLs.
    • Producers and Consumers: Producers can produce data only to topics visible within their domain. Consumer runtime data tables, including Kafka Streams, Subscriptions, Partition Groups, Consumer Statistics, and Unprocessed Messages, are domain separated. Kafka streams and associated subscription groups are created and managed within the domain context to ensure correct data import per domain.
    • Schema Management: Schema-related tables, including Stream Connect Schemas and Schema Registries (both standalone and Confluent versions), are domain separated to maintain schema isolation and proper governance across domains.

    Key Outcomes

    • Secure multi-tenant support by isolating data and processes per domain, enabling service providers to manage multiple customers in a single instance effectively.
    • Fine-grained access control to Kafka topics and stream data based on domain membership and ACLs, improving data security and compliance.
    • Consistent domain-aware business logic and administrative control, such as applying domain-specific rules (e.g., requiring comments on record closure for some tenants but not others).
    • Improved operational clarity and data governance by aligning message replication, stream processing, and schema management with domain boundaries.

    Domain separation is supported for Stream Connect. Domain separation enables you to separate data, processes, and administrative tasks into logical groupings called domains. You can control several aspects of this separation, including which users can see and access data.

    Support level: Standard

    • Includes all aspects of Basic level support.
    • Application properties are domain-aware as needed.
    • Business logic: The service provider (SP) creates or modifies processes per customer. The use cases reflect proper use of the application by multiple SP customers in a single instance.
    • The instance owner must configure the minimum viable product (MVP) business logic and data parameters per tenant as expected for the specific application.

    Sample use case: An admin must be able to make comments required when a record closes for one tenant, but not for another.

    For more information on support levels, see Application support for domain separation.

    Message replication

    Records in the following Stream Connect Message Replication tables are domain separated.
    • Message Replications [sys_sc_message_replication]
    • Channel Replications [sys_sc_channel_replication]
    • Message Replication Statistics [sys_sc_channel_replication_metric]
    • Kafka Topic Replications [sys_kafka_topic_replication]

    The domain for a message replication record is specified in the record's sys_domain field.

    For channel replication and replication metrics records, the domain is determined by the domain_master attribute. The domain_master attribute uses a reference field's domain to determine the domain for the current record. A channel replication record gets its domain from the referenced message replication record. A replication metrics record gets its domain from the referenced channel replication record.

    A Kafka topic replication record has the same domain as its associated channel replication record because the Channel Replications table is a parent of the Kafka Topic Replications table.

    Topic namespaces

    You can use topic namespaces to configure which domains can access a Kafka topic on a domain-separated instance. In ServiceNow, Kafka topics are linked to a namespace. Namespaces can be linked to ServiceNow domains. When a namespace is assigned to a specific domain, all the topics created with that namespace are also assigned to the namespace's domain. Users can only see and interact with the topics they have access to, based on domain visibility and access control lists (ACLs).

    For more information, see Managing namespaces and topics in Hermes.

    Producers and consumers

    Stream Connect producers and consumers are also domain separated. For producers, you can produce data only to topics that are visible to the domain.

    For consumers, all the tables used to keep runtime data are domain separated. These tables include the following.
    • Kafka Streams [sys_kafka_stream]
    • Kafka Subscriptions [sys_kafka_subscription]
    • Kafka Subscription Partition Groups [sys_kafka_partition_group]
    • Kafka Consumer Statistics [sys_kafka_consumer_statistics]
    • Kafka Unprocessed Messages [sys_kafka_unprocessed_messages]

    Domain users can create domain-specific Kafka streams with the topics that are visible to the domain. When activating a Kafka stream, the subscription and partition groups are created within the same domain. At runtime, the domain of the partition group is set by the consumer thread so that all the data is imported to the correct domain. For more information on producers, consumers, and Kafka streams, see Using Stream Connect for Apache Kafka.

    Schema management

    Tables for schemas and schema registries are domain separated. These include the following.
    • Stream Connect Schemas [stream_connect_schema]
    • Standalone Stream Connect Schema [standalone_stream_connect_schema]
    • Confluent Stream Connect Schema [confluent_stream_connect_schema]
    • Stream Connect Schema Registry [stream_connect_schema_registry]
    • Confluent Stream Connect Schema [confluent_stream_connect_schema]
    • Standalone Stream Connect Schema [standalone_stream_connect_schema]