Microsoft Azure Log Analytics data input configuration fields
Summarize
Summary of Microsoft Azure Log Analytics data input configuration fields
This guide explains the configuration fields available when setting up Microsoft Azure Log Analytics data inputs in ServiceNow. It helps customers integrate Azure log data into their ServiceNow instance using MID Servers or MID Server clusters, enabling automated log ingestion and monitoring through Health Log Analytics.
Show less
Basic Configuration
- Name and Description: Define a unique name and optional description for the data input.
- Execute on: Choose between a specific MID Server or a MID Server cluster. Only MID Servers supporting basic authentication are listed; mTLS-enabled servers are excluded.
- MID Server/MID Server Cluster: Specify the exact MID Server or cluster where logs will be pulled. Clusters provide failover capability, transferring data input tasks to another server if one fails. Each MID Server supports up to 10 concurrent data inputs by default, adjustable in MID Server properties.
- Service Instance: Bind the data input to an existing operational service instance. If none exists, create one and assign configuration items before binding.
- Health Log Analytics automatically enables log ingestion for selected MID Servers if not already active.
Read-Only Status Fields
- Status of the data input (active or disabled)
- Transport protocol used for streaming
- Timestamp when the data input stopped or failed
- Number of log sources created
- Time of last received log
- Streaming error messages, if any
Query Settings
These settings define how log data is queried from Microsoft Azure Log Analytics:
- From: Specify the starting date/time for log retrieval; important to avoid congestion when querying large historic datasets.
- Data source name: The Azure Log Analytics table name to fetch logs from, e.g., ContainerLog.
- Event time property name: The field indicating event timestamps, e.g., TimeGenerated.
- Max documents per query: Limits rows per query (default 500).
- Columns to select: Specify columns to retrieve; ignored if a custom query is provided.
- Log query: Allows a custom JSON-formatted query to override other query settings (except the From field). This enables tailored data retrieval scenarios.
Transport Configuration
- Redirect URL: The Azure redirect URI for authorization, required to establish secure connections.
- Azure service principal credentials: Use a Client Secret to authenticate with Azure resources; Azure Enterprise Agreement credentials are not required.
- Workspace Id: The Customer ID for calling the Azure Log Analytics REST API.
Advanced Configuration
- Event Processor workers: Number of concurrent workers processing event batches (default 4).
- Queue Size: Size of the queue for event processing workers (default 5).
- Sub sample drop ratio and receive ratio: Settings to reduce or control the volume of fetched events by discarding some events in batches (default -1 disables this).
- Character encoding: Encoding for incoming data, default is UTF-8.
- Sleep interval: Seconds to wait before querying again after no events are returned (default 60 seconds).
- Polling interval: Time in seconds between polling attempts (default 0).
- Drop if queue is full: Option to discard logs when MID Server load is high (default false).
Practical Implications for ServiceNow Customers
By carefully configuring these fields, customers can efficiently stream Azure log data into their ServiceNow environment, leveraging MID Servers for failover and load management. Custom queries enable tailored data ingestion, while advanced settings help optimize performance and resource utilization. This integration supports proactive health monitoring and incident management by providing timely and relevant Azure log information within ServiceNow.
Description of the fields on the Microsoft Azure Log Analytics data input configuration form.
Basic configuration
| Field | Description |
|---|---|
| Name | Name of the new data input. This field is required. |
| Description | Description of the data input. |
| Execute on | Option to select whether to use a specific MID Server or a MID Server cluster. This field is required. |
| MID | (Only when the Execute on field is set to Specific MID Server) MID Server to which log data from Microsoft Azure Log Analytics is pulled.Note: This field is required.
|
| MID Server Cluster | (Only when Execute on is set to Specific MID Server cluster.) The MID Server cluster to which the log data is pulled. This field is required.The data input runs on a single MID Server in the cluster until that MID Server fails. The system then moves all the data input tasks to the next available MID Server in the cluster according to the configured order. Note: For more information about MID Server clusters, see Configure a MID Server cluster.
|
| Service instance | The service instance to which to bind the log data. Note: This field is required.If no relevant service instance exists, Create an service instance and add CIs to it. Set the status of the new service instance to Operational. |
| Field | Description |
|---|---|
| Status | Status of the data input. |
| Transport | Protocol used to stream the log data. This data input uses Microsoft Azure Log Analytics to stream log data to your instance. |
| Disabled since | The time when the data input stopped or failed. |
| Sources count | The number of log sources this data input has created. |
| Last log time | The time when the last log streamed in the data input. |
| Error message | The streaming error. This field is populated automatically. It displays only when a streaming error has occurred. |
| Field | Description | Example |
|---|---|---|
| From | Starting date and time for reading the data. Data older than this date and time is not read. Note:
Setting this value to a past date might require the system to read large amounts of data, causing
congestion. This field is required. |
Now -1 week |
| Data source name | The name of the table in Microsoft Azure Log Analytics where the data input fetches the log data. For more information, see the View table information section in the Microsoft Azure documentation. This field is required. |
ContainerLog |
| Event time property name | The Microsoft Azure Log Analytics field in which to detect the event time. This field is required. |
TimeGenerated |
| Max documents per query | The highest number of rows retrieved in each query. | 500 |
| Columns to select | Comma-separated list of column names to return. Note: This field is ignored when you provide a custom query. |
LogEntry,LogEntrySource |
| Log query | A custom query for receiving log data from Microsoft Azure
Log Analytics. The settings in this field override those in all other fields on the Query settings tab except the From field. If the Log query field is empty, Health Log Analytics generates the query using the values set in the other fields. For the custom query, use the following JSON format:
|
|
| Field | Description |
|---|---|
| Redirect Url | The redirect URL of the access log application. The URL refers to the Microsoft Azure redirect_uri authorization property. For more information, see the Authorization code URL (GET request) section in the Microsoft Azure documentation. This field is required. |
| Azure service principal credentials | The credentials used to access Microsoft Azure
resources. Choose Client Secret from the drop-down list. Note:
You do not need Azure Enterprise Agreement
(EA) credentials. This field is required. |
| Workspace Id | The Customer ID used to call the Microsoft Azure Log Analytics REST API. |
Advanced configuration
| Field | Description | Default value |
|---|---|---|
| Event Processor workers | The number of concurrent event processing workers, where each worker processes a batch of events independently. | 4 |
| Workers Queue Size | The queue size of the Event Processor workers. | 5 |
| Sub sample drop ratio | The number of events to batch together, out of which one will be discarded. This setting is used to reduce the number of fetched events. | -1 |
| Sub sample receive ratio | The number of events to batch together, out of which all but one will be discarded. This setting is used to decrease the number of received events. | -1 |
| Character encoding | The character encoding for this data input. | UTF-8 |
| Sleep interval | The interval, in seconds, to wait before querying again after a query has returned no events. | 60 |
| Polling interval | The interval, in seconds, to wait before polling for new events. | 0 |
| Drop if queue is full | Option for selecting to discard logs if there is a load on the MID Server. | False |