Discovery Credentials

carloschaluisan
Kilo Contributor

While running our Discovery Schedules, a significant amount of authentication / credential issues are being generated. The errors with the highest occurrence are:

  • Connection failed to WMI service. Error: Permission denied
  • SSHCommand: No valid credential found for types [SSH Password,SSH Private Key]
  • Failed to access target system. Please check credentials and firewall settings on the target system to ensure accessibility: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))

Here is what we've done so far:

  • Confirmed the usernames and passwords in the Credentials table are correct Confirmed the MID Servers are up.
  • Firewalls in the Host Machines where the MID servers are installed are disabled
  • Set mid.use_powershell = true in each MID Server
  • Set mid.powershell.use_credentials = true in each MID server
  • Set mid.powershell.local_mid_service_credential_fallback = false in each MID server
  • Manually ran WMI (gwmi) queries and tested SSH from the Host servers where the MID servers are installed and confirmed access is granted and data is returned.

Any troubleshooting suggestions would be greatly appreciated...

20 REPLIES 20

tim_broberg
ServiceNow Employee
ServiceNow Employee

Well, I'm an expert on SSH but I live in fear of Windows credentials, so I'll help where I can.



  1. If your version of the product supports it, you can try 'test credential' on the credential page. If this works but normal operation doesn't, perhaps you have so many credentials to try that the server thinks you're trying to hack it and throws you out. Reduce the order on the credential you need so that it gets tried sooner.
  2. You may find that your server is demanding keyboard_interactive authentication. Older versions of the mid server did not try KBI once they have tried password. You can try setting mid.ssh.use_keyboard_interactive = true.
  3. On you mid server, you can set mid.ssh.debug = 10.11.12.13 (change to your IP address), try to connect, and then go back to the mid, hit grab logs, refresh until the inputs come back, and look at the agent log. This will contain all the gory details of what happened in ssh. Search for "ssh-userauth" to get in the neighborhood.
  4. Consider looking at the logs on the server, which are often in /ver/log/secure, to see if he has anything interesting to say about the failures. For openssh, you can set the loglevel in /etc/ssh/sshd_config to debug3 and restart sshd to get a ridiculous amount of detail.
  5. I would highly recommend public key authentication over passwords because of the security benefits, but it also bypasses any possible keyboard_interactive issues.


Hope this helps. Feel free to send me that agent log if you need somebody to read the tea leaves.



Hopefully somebody braver can address the Windows issues.


    - Tim.


carloschaluisan
Kilo Contributor

I should add that we also tested the Windows Credentials stored in the table by RDP into multiple windows boxes for which we are getting WMI errors.


I would suggest then to do a quick discovery specially on the Windows box you tested RDP on and verify you get passed the classification phase which means your credentials worked. If this does not work, I would retype the password in, or if you are on Helsinki, there is a test credentials option now available from the credential form.


Thanks Glenn.   I am in Helsinki.   Credential tests fail for both Windows and SSH.   But when I run the same commands manually, using the same credentials, form the servers hosting the MID (ex. gwmi) I am able to get the info I am querying for...


I have seen the Test Credentials feature fail, but when actual discovery is run it works. So if you do a quick discovery on a single target that you believe you have credentials for, does that discovery complete successfully or fail during the classification phase?