Having a background in System Engineering helps when it comes to administering MID Servers. I wrote a comment the other day about how to configure the wrapper program to auto-restart the JVM based on specific error messages. MID Server socket timeout (HTTP 202) errors
Sometimes it is necessary to start or restart the MID Server. Looking at the MID Server form, you will find the provided UI Action: "Restart MID" (green arrow below). This UI Action sends a command to the ecc_queue for that MID Server (ecc_agent) to pick up and execute just like any other probe or task.
But what if the MID Server isn't picking up work from the ecc_queue? If something has gone awry, networking issues, storage problems - including brief disruptions - the MID Server application can end up in a bad state where the work isn't getting picked up from the ecc_queue. Symptoms of this would be log statements in the agent\logs\agent.log.0 file saying that xml is being enqueued or even SEVERE errors contacting your SN instance. In these cases and others, there is good reason to restart the Windows Service entirely.
Remoting into the Windows host is time consuming, should require some level of access control including group memberships that must be maintained for employees and contractors and is just all around tedious in nature. You could use the sc.exe command in Windows and structure the command to remotely restart the service from your workstation. I've used that for a good while and it's not a bad workaround. Writing the wrapper script for doing it inside Cygwin was just difficult enough to make things interesting. This solution has portability problems for people not used to using Cygwin and it still requires group memberships and access controls for your employees and contractors to access the MID Server host and restart services.
Enter the "Brute Restart MID" UI Action.
Thanks to PowerShell Probe Script Utility and a standard Windows Service naming convention (see line 4 below), it is possible to script this as a UI Action. Be sure that you have your condition set on this to limit access to your ITOM admins.
var target_agent_name = current.name.replace(/'/g, "\\'");
var target_host = current.host_name;
var ps_script = 'restart-service -inputobject (get-service -computername '+ target_host
+' -name snc_mid.'+ target_agent_name + ')';
var orch_host = gs.getProperty('mid.server.rba_default', 'NONE');
if (orch_host == 'NONE'){
gs.addErrorMessage('Please set mid.server.rba_default sys_property.');
} else {
var powerShellProbe = new PowershellProbeES(orch_host);
powerShellProbe.setScript(ps_script);
powerShellProbe.create();
gs.addInfoMessage('Remote Service Restart Issued.');
}
action.setRedirectURL(current);
I also added a Business Rule to ecc_queue to move attachments from the "Grab MID logs" action to the MID Server record.
Together these small enhancements simplify the routine tasks associated with MID Server administration. They also allow us to keep access to the MID Server host machines more tightly controlled, which is good practise anyways. They also allow those of us with access to stay off our VPN clients which is a definite win.
3 Comments
