Microsoft Azure Log Analytics data input configuration fields

  • Release version: Yokohama
  • Updated January 30, 2025
  • 5 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Microsoft Azure Log Analytics Data Input Configuration Fields

    This guide provides detailed descriptions of the configuration fields required to set up Microsoft Azure Log Analytics data inputs in ServiceNow. These configurations enable you to stream log data from Microsoft Azure Log Analytics into your ServiceNow instance using MID Servers or MID Server clusters, ensuring reliable data ingestion and integration with your operational environment.

    Show full answer Show less

    Basic Configuration

    • Name & Description: Define a unique name and optional description for the data input.
    • Execute On: Choose whether to run the input on a specific MID Server or a MID Server cluster.
    • MID Server Selection: Select MID Servers that support basic authentication only (mTLS is not supported). Log ingestion must be enabled on these MID Servers; if not, Health Log Analytics will enable it automatically. Note that a single MID Server supports up to 10 concurrent streaming data inputs by default.
    • MID Server Cluster: Select failover clusters composed exclusively of basic authentication MID Servers. Clusters provide failover protection by moving tasks to the next available MID Server if the active one fails.
    • Service Instance: Bind the log data to a relevant service instance set to Operational. If none exists, create one and add configuration items (CIs) accordingly.
    • Read-only Status Fields: Monitor the status, transport protocol, last log time, source count, and any streaming errors directly from the data input form.

    Query Settings

    Configure how logs are queried from Microsoft Azure Log Analytics:

    • From: Specify the start date and time for reading data; setting this to a past date may cause performance impacts.
    • Data Source Name: Identify the Azure Log Analytics table (e.g., ContainerLog) to fetch logs from.
    • Event Time Property: Indicate the field in Azure logs that holds the event timestamp (commonly TimeGenerated).
    • Max Documents per Query: Limit the number of rows retrieved per query (default is 500).
    • Columns to Select: Define comma-separated columns to return; ignored if a custom query is used.
    • Log Query: Optionally input a custom JSON-formatted query to override other query settings, allowing tailored log retrieval.

    Transport Configuration

    • Redirect URL: Set the Microsoft Azure redirecturi used for authorization.
    • Azure Service Principal Credentials: Provide credentials (Client Secret) to access Azure resources; Azure Enterprise Agreement credentials are not required.
    • Workspace ID: Enter the Customer ID that identifies the Azure Log Analytics workspace for API calls.

    Advanced Configuration

    Fine-tune performance and event processing:

    • Event Processor Workers: Number of concurrent workers processing event batches (default 4).
    • Workers Queue Size: Size of the queue for processing events (default 5).
    • Sub Sample Drop and Receive Ratios: Control event batching and reduction to manage volume; defaults are -1 (disabled).
    • Character Encoding: Encoding used for incoming data, typically UTF-8.
    • Sleep Interval: Time in seconds to wait before re-querying when no new events are found (default 60).
    • Polling Interval: Interval to poll for new events; zero means immediate polling.
    • Drop If Queue Is Full: Option to discard logs when MID Server load is high, which defaults to false (do not drop).

    Practical Benefits for ServiceNow Customers

    By correctly configuring these fields, ServiceNow customers can reliably ingest Azure log data for real-time monitoring, troubleshooting, and analytics within the ServiceNow platform. The use of MID Server clusters ensures high availability and failover support, while customizable queries and advanced settings allow the tuning of data volume and processing performance to suit organizational needs.

    Description of the fields on the Microsoft Azure Log Analytics data input configuration form.

    Basic configuration

    Field Description
    Name Name of the new data input. This field is required.
    Description Description of the data input.
    Execute on Option to select whether to use a specific MID Server or a MID Server cluster. This field is required.
    MID

    (Only when the Execute on field is set to Specific MID Server)

    MID Server to which log data from Microsoft Azure Log Analytics is pulled.
    Note:
    • You can select only MID Servers that support basic authentication. MID Servers that support mTLS are not listed.
    • The default maximum number of data inputs streaming logs to a single MID Server is 10. You can modify this number in the MID Server properties.
    • If log ingestion is not enabled for the selected MID Server, Health Log Analytics enables it automatically.
    This field is required.
    MID Server Cluster

    (Only when Execute on is set to Specific MID Server cluster.)

    The MID Server cluster to which the log data is pulled. This field is required.

    The data input runs on a single MID Server in the cluster until that MID Server fails. The system then moves all the data input tasks to the next available MID Server in the cluster according to the configured order.

    Note:
    • Health Log Analytics supports only failover MID Server clusters. In these clusters, multiple MID Servers are grouped together for failover protection. When selecting a cluster from the data input or integration form, the MID Server clusters list displays only failover clusters.
    • The MID Server cluster must include only MID Servers that support basic authentication. mTLS is not supported for log ingestion.
    • Log ingestion must be enabled for each MID Server in the cluster. If log ingestion is not enabled for the active MID Server, Health Log Analytics enables it automatically.
    • The default maximum number of data inputs or integrations streaming logs to a single MID Server is 10. A cluster passes capacity validation if it contains at least one MID Server with fewer than 10 data inputs or integrations running on it, even when that MID Server is down.
    For more information about MID Server clusters, see Configure a MID Server cluster.
    Service instance The service instance to which to bind the log data.
    Note:
    If no relevant service instance exists, Create an service instance and add CIs to it. Set the status of the new service instance to Operational.
    This field is required.
    The following fields show read-only information:
    Field Description
    Status Status of the data input.
    Transport Protocol used to stream the log data.

    This data input uses Microsoft Azure Log Analytics to stream log data to your instance.
    Disabled since The time when the data input stopped or failed.
    Sources count The number of log sources this data input has created.
    Last log time The time when the last log streamed in the data input.
    Error message The streaming error.

    This field is populated automatically. It displays only when a streaming error has occurred.
    Table 1. Query settings tab

    Health Log Analytics uses the values set in these fields to generate the query for receiving log data from Microsoft Azure Log Analytics. The Log query field enables you to configure a custom query
    Field Description Example
    From Starting date and time for reading the data. Data older than this date and time is not read.
    Note:
    Setting this value to a past date might require the system to read large amounts of data, causing congestion.

    This field is required.

    		 Now -1 week
    Data source name The name of the table in Microsoft Azure Log Analytics where the data input fetches the log data. For more information, see the View table information section in the Microsoft Azure documentation.

    This field is required.

    		 ContainerLog
    Event time property name The Microsoft Azure Log Analytics field in which to detect the event time.

    This field is required.

    		 TimeGenerated
    Max documents per query The highest number of rows retrieved in each query. 500
    Columns to select Comma-separated list of column names to return.
    Note:
    This field is ignored when you provide a custom query.
    		 LogEntry,LogEntrySource
    Log query A custom query for receiving log data from Microsoft Azure Log Analytics.

    The settings in this field override those in all other fields on the Query settings tab except the From field. If the Log query field is empty, Health Log Analytics generates the query using the values set in the other fields.

    For the custom query, use the following JSON format:

    {"query": "query | where TimeGenerated > %s | take 500"}

    		 
    {
"query":"ContainerLog
                    | where LogEntry contains 'cartservice'
                    | where TimeGenerated > %s
                    | take 500",
"workspaces": ["defaultworkspace-3ab145ff-f9cd-433f-8533-d1b1ee24aee6-eus"],
"project": ["TimeGenerated", "LogEntry", "LogEntrySource"]
}
    Table 2. Transport tab
    Field Description
    Redirect Url The redirect URL of the access log application.

    The URL refers to the Microsoft Azure redirect_uri authorization property. For more information, see the Authorization code URL (GET request) section in the Microsoft Azure documentation.

    This field is required.
    Azure service principal credentials The credentials used to access Microsoft Azure resources.

    Choose Client Secret from the drop-down list.

    Note:
    You do not need Azure Enterprise Agreement (EA) credentials.

    This field is required.
    Workspace Id The Customer ID used to call the Microsoft Azure Log Analytics REST API.

    Advanced configuration

    Table 3. Advanced configuration form
    Field Description Default value
    Event Processor workers The number of concurrent event processing workers, where each worker processes a batch of events independently. 4
    Workers Queue Size The queue size of the Event Processor workers. 5
    Sub sample drop ratio The number of events to batch together, out of which one will be discarded. This setting is used to reduce the number of fetched events. -1
    Sub sample receive ratio The number of events to batch together, out of which all but one will be discarded. This setting is used to decrease the number of received events. -1
    Character encoding The character encoding for this data input. UTF-8
    Sleep interval The interval, in seconds, to wait before querying again after a query has returned no events. 60
    Polling interval The interval, in seconds, to wait before polling for new events. 0
    Drop if queue is full Option for selecting to discard logs if there is a load on the MID Server. False