ITOM Discovery Best Practices for Linux Servers

[BillMartin]

In today’s dynamic IT environments, maintaining an accurate and reliable Configuration Management Database (CMDB) is critical for effective IT Operations Management (ITOM). For organizations leveraging ServiceNow ITOM Discovery to manage their Linux servers, applying best practices ensures efficient discovery, optimal performance, and accurate data collection. This article highlights key strategies and tips for discovering Linux servers using ServiceNow.

Preparation: Set the Foundation for Successful ServiceNow Discovery

Inventory and Scope Definition

Start by identifying all Linux servers in your environment and defining the discovery scope. Determine which subnets and IP ranges are relevant, and exclude unnecessary ranges to minimize noise and optimize performance.

Network Configuration

Proper network access is essential for successful discovery. Ensure the following ports are open:

• SSH (Port 22): For secure Linux server communication.
• ICMP (Ping): For basic device availability checks.

Work with your network and security teams to establish these configurations without compromising security.

Secure Credentials

Create a dedicated Linux Service Account with the following permissions:

• Ability to execute basic commands such as uname, df, lsblk, and lsof.
• Read access to configuration files like /etc/passwd, /etc/group, and /proc/mounts.

Store credentials securely in ServiceNow using the Credential Store, and test them before initiating discovery.

How to Conigure Discovery in ServiceNow

MID Server Placement

The MID server acts as the bridge between ServiceNow and your infrastructure. Ensure the following:

• Place the MID server close to the target Linux servers to reduce latency.
• Assign sufficient resources to handle multiple concurrent discovery tasks.

Enable Linux Discovery Patterns

ServiceNow provides out-of-the-box Linux Discovery Patterns that automate the collection of data such as:

• Hostname and OS version
• CPU, memory, and disk details
• Running processes and installed packages

Ensure these patterns are enabled in the Discovery Patterns module for efficient discovery.

Test Credentials

Use the Credential Test feature in ServiceNow to verify SSH connectivity and permissions for your Linux servers. This step reduces errors during the discovery process.

Execution: Running Discovery Effectively

Schedule Discovery

Automate discovery by creating Discovery Schedules for Linux servers. Adjust the frequency based on the rate of infrastructure changes:

• Dynamic environments: Daily
• Static environments: Weekly or bi-weekly

Incremental Discovery

Start with horizontal discovery to identify devices and servers. Then, enable vertical discovery to collect detailed application and process-level information.

Optimize Data Collection

Refine discovery by:

• Excluding irrelevant IP ranges or devices using IP Range Filters.
• Suppressing discovery of non-essential virtual interfaces.

 

Post-Discovery Validation and Optimization

Validate CMDB Data

Regularly validate the accuracy of your CMDB by cross-referencing discovery results with other inventory systems. Pay attention to:

• Completeness: Ensure all Linux servers are discovered.
• Accuracy: Verify collected data matches the actual state of the servers.

Monitor MID Server Performance

Check MID server logs and performance dashboards to identify bottlenecks. Optimize probe configurations to improve discovery speed and efficiency.

Deduplicate Records

Enable reconciliation rules to merge duplicate records and ensure a clean CMDB.

Security and Compliance

Credential Management

Secure credentials in ServiceNow’s Credential Store, ensuring they are encrypted and accessible only by authorized users.

Logging and Monitoring

Enable detailed discovery logs to maintain an audit trail. Monitor logs for anomalies, such as failed discoveries or unauthorized access attempts.

Regulatory Compliance

Ensure discovery processes comply with organizational and legal security policies, such as GDPR or HIPAA, by restricting access to sensitive data.

Beyond Discovery: Leveraging Service Mapping

Integrate discovery results into Service Mapping to visualize application dependencies and understand the impact of server performance on business services. This integration enhances visibility and improves incident and problem management.

Conclusion

Effective discovery of Linux servers using ServiceNow ITOM Discovery requires careful planning, secure configurations, and ongoing optimization. By following these best practices, you can maintain an accurate and reliable CMDB, ensuring better IT operations and service delivery.

If you have additional tips or questions about ITOM Discovery for Linux servers, feel free to share them in the comments. Let’s collaborate to enhance our collective expertise!

Join the discussion in the ServiceNow Community to share your insights and learn from others!

[Phoomipat]

After setting up SSH Private Key Credentials, I tested it but still encountered the error: "Authentication failed."

I double-checked all the prerequisites with the customer, and everything seems to be in place:

1. The target devices have a service account for the MID Server — confirmed.
2. Port 22 is open on the target devices — confirmed.
3. The target IP has sudo command access — confirmed.

How can I determine the best way to discover Linux devices or identify the correct type of credentials to use for my customer? Additionally, how can I pinpoint which areas of the Linux discovery process to investigate or recommend the customer to focus on for resolving the issue?

[BillMartin · Accepted solution]

Hi  @Phoomipat  ,
 

To identify and resolve the issue, I recommend the following steps to verify and pinpoint where the error lies. Additionally, using a network analysis tool like Wireshark can be invaluable for debugging each transaction on the network. It provides detailed insights into the communication occurring between your system and the target server.

Your problem is complex, as it depends on multiple factors, including the configurations and security parameters of your current environment. You will likely need to communicate with various stakeholders or personas within your client’s organization who are responsible for different configurations, such as network administrators, security teams, and system administrators.

Ideally, all these configurations are well-documented, but in practice, they often are not. As a result, resolving the issue will require a great deal of patience, collaboration, and persistence as you piece together the necessary information and troubleshoot each layer of the system.

 Here it goes: 

Ensure the private key is in the correct format (e.g., OpenSSH or PEM). 
Confirm the private key matches the public key configured on the target Linux server
Test the Key Outside ServiceNow
From the machine hosting the MID Server, test the private key with the target Linux server:
If this fails, it indicates an issue with the key itself, the user, or the server configuration.
Verify Key Permissions
The private key file on the MID Server host must have secure permissions:
Ensure that the ~/.ssh/authorized_keys file on the Linux server has proper permissions:
Check the MID Server Configuration
Ensure the MID Server has read access to the private key file if you are specifying it locally. If using a stored credential in ServiceNow, verify:
* The correct user is specified.
* The private key was entered correctly (no extra spaces or formatting issues).
Use the Correct Username
Confirm the username in the SSH credentials matches the user associated with the public key on the Linux server. Test the connection manually:
Check SSH Configuration on the Target Server
Ensure the target server's SSH daemon (sshd) is configured to accept public key authentication. Verify the following in /etc/ssh/sshd_config:
Restart the SSH service after making changes:
Verify Network Connectivity
Ensure the MID Server can reach the target server on port 22:
If connectivity fails, check for firewalls or network restrictions.
Enable Debug Logs on the Linux Server
Review logs on the Linux server to identify authentication issues:
Look for errors such as:
* Permission denied (publickey).
* Authentication refused: bad ownership or modes for file.
Review ServiceNow Logs
On the MID Server, review the agent logs for more detailed error messages:
* agent0.log
Look for specific errors related to credential usage or SSH connectivity.
Test Alternative Credentials
If the private key continues to fail, try setting up and testing password-based credentials to verify basic connectivity and authentication.
Regenerate SSH Keys (if necessary)
If all else fails, regenerate the key pair:
1. On the MID Server machine or a secure machine:
2. Copy the new public key to the target server
Verify ServiceNow MID Server Credential Setup
Go to Discovery > Credentials in ServiceNow and edit the SSH Private Key Credential. Confirm:
* The private key is correctly pasted (avoid spaces and empty lines).
* The correct username is associated with the key.
By systematically following these steps, you should be able to identify and resolve the root cause of the "Authentication failed" error. If the issue persists, consider reaching out to ServiceNow support for further assistance.