When implementing Discovery, Service Mapping (aka ServiceNow Visibility) or Orchestration tasks through Integration Hub into your company, one guy (often one guy from the security team) asked why your credentials need so many privileges and how do you plan to secure them.
In this article, we'll try to reassure the SecOps Team and give you some clues to secure them following state-of-the-art best practices.
Why does Discovery need admin privileges ?
Accounts used by Discovery need to connect to a wide range of servers or devices and require admin privileges (sudo privileges) to discover those resources properly.
- List of commands required for ServiceMapping are documented here.
- List of commands required for Discovery for SSH Probe are documented here
Even if you restrict the privileges as much as possible, these accounts still have a lot of privileges.
Credentials default security limits and solution ?
ServiceNow protects and encrypts all credentials and passwords with recognized security principles and practices implemented between MID Servers and Cloud instance, but :
- Someone with development skills can easily reverse password encryption with admin role, or through a rogue web service deployment.
- ServiceNow administrators from your company have also exposure to social engineering and if one of them is hacked, your company can run into a lot of trouble.
- Another entrypoint are ServiceNow instances dedicated to DEV, SIT or UAT activities where accounts and credentials management can be less secured
- Service Maps cannot be easily moved from Dev instance to Production, what often means using production credentials on non-production instances to iterate during the Service Maps creation process
- ...
Now, how can we mitigate these risks and secure secrets or credentials with elevated privileges ?
- TL;DR : Do not store secrets/passwords into ServiceNow instance, just use an external credential storage
External Credential Storage
With External Credential Storage, ServiceNow will delegate the credentials to an external solution. Why is it so interesting ?
- You can choose a tool specialized into secure secret storage
- You can host this solution on premise or into your private Cloud on AWS, Azure, GCP and keep it under your control
- ServiceNow instances will not store any credentials, only identifiers which can't be used retrieve linked credentials
- On-premise MID Servers will query the external credential storage each time they need them, so no risk of reverse attack with admin access or rogue web service, and no secret sprawl between instances.
The following diagram presents architecture without External Credential Storage :
The following diagram presents the same architecture but introduce an external credential storage to secure secrets
As you can see on this diagram, Hashicorp Vault is the solution used to manage and store secrets securely outside ServiceNow instance.
Hashicorp Vault
We will not explain how to configure Hashicorp Vault as there are a lot of How-To available on the Internet, but we will list the features that increase the security of your precious credentials.
Use mutual TLS
Communication between MID Server has to be done over HTTPS. But to ensure the MID Server is the one who it pretends to be, you have to configure mutual TLS by setting the certificate authorities for clients
Configure AppRole
AppRole is a best practice for an application (ServiceNow via the MID Server) querying Hashicorp Vault.
AppRole provides a Role ID and a Secret ID to authenticate against Vault.
We suggest you to create a MID Server Property for the Role ID.
For the Secret ID, create a parameter into the configuration file of the MID server and encrypt it.
On top of that, you can configure the AppRole to be usable only by the MID Servers by restricting the access with bound CIDR
All these security optimization required MID Servers host hardening to be congruent.
Use Dynamics Credentials
Vault can store static credentials into the KV store. But for accounts with elevated privileges, make the last miles to achieve great security and as a best practice use dynamics credential when possible.
For example with Active Directory account, you can use password rotation. To illustrate, imagine password rotation of admin account used for Discovery operations every 8 hours. Nobody except Mid Server can access or read it. You just have to schedule the rotation action and if you have AD replication across regions, to take the password replication process into account.
Another example with databases account. You can create account or elevate the privileges on demand to avoid long life password. Just be careful to Discovery process, because the probes queries a few times (up to 10 or 15) the credential to call a database. So you have to handle that to avoid creating 10 or 15 accounts on the database at each discovery.
How MID Server can communicate with Vault ?
The MID Server needs to communicate with Vault to ask for the credentials.
To do that you can develop your own external credential resolver as documented here and address all the points explained previously.
Or if you prefer a turnkey solution, JFLA Consulting has created Vault Connect which
- will save you a few (a lot maybe) days of development, maintenance and industrialization,
- implements all the main features explained into this article in a secured by design mindset,
-
- provides support for password rotation during discovery
- provides caching
- handles multiple datacenters and authentication methods
- ...
- offers a day-1 solution with easy configurations to map your credentials from ServiceNow with the one inside Vault,
- has been battle-tested on large infrastructures with thousands of servers or devices.
Conclusion
Whatever the solution you will choose, please consider to use an external credential storage and a secure by design resolver to store and manage your ITOM credentials.
