Migration from J2SSH to SNCSSH

[tim_broberg]

Eureka adds a new SSH library, SNCSSH or "ServiceNow SSH,"   which is gradually replacing the older J2SSH library or "Legacy SSH."

J2SSH limitations:

1. no longer being supported by the open source community
2. the set of negotiable algorithms is steadily aging, for example there is no support for diffie-hellman-group14-sha1 and aes256-ctr
3. large number of threads per connection strains server resources
4. limited number of simultaneous probes per IP, by default three
5. maintainability of the J2SSH code is less than optimal
6. poor resource locking causes stalls during connection
7. scripts are scp'ed to the target system in a tmp directory, which can fail when no temp space is available

New features of SNCSSH:

1. nio-based architecture greatly reduces the number of threads
2. improved error messages
3. "deferred" debug logging, collects full debug information for each connection, displaying it only when that connection encounters an error
4. number of simultaneous probes limited only by server's MaxSessions, by default seven
5. where possible, scripts are handled as bourne-shell functions, reducing the number of operations required and removing the requirement to execute scripts from the temp directory

Terminal vs channels

Under the hood, sncssh uses ssh connections in a different way from j2ssh, which may result in some subtle differences in behavior.

A "terminal" is like the ssh connection you're probably used, a persistent connection where you type one character at a time, and watch for a command prompt to know it's your turn to type.

A "channel" is an internal feature of SSH sessions that allows multiple operations to be conducted over the same session, including execution of commands. This often goes by the term, "multiplexing" in ssh implementations.

j2ssh

• creates a pool of terminals, with one terminal used per probe
• each terminal has its own session to the server in question, so it's a heavy object
  
• commands are executed by sending a sequence of characters through the input stream of the terminal, and results are gleaned from the output stream.
• state from one probe may remain in the terminal when a subsequent probe is run. (This is why you cannot end a probe with "exit." It closes the terminal.)

sncssh

• creates a new channel for each operation run
• all the channels for the same host share a session, so it they are light objects
• commands to be executed are handled as discrete requests to the channel, one command per channel
• the lifetime of the channel is a single command, with no state carried over from previous commands

Selection of sncssh vs j2ssh

SNCSSH is enabled by default for new customers in Eureka and Fuji. Existing customers retain the same j2ssh.

You may select which library you prefer for the whole instance, for individual mid servers, and probe by probe.

Boolean attributes enabling sncssh, in order of descending precedence:

Probe - use_snc_ssh

Mid Server - (Discovery | Orchestration) / Mid Servers / (Your individual mid server) / Configuration Parameters / mid.ssh.use_snc

Instance - Mid Server / Properties / mid.property.ssh.use_snc

Detection of sncssh

When sncssh is in use, the probe parameter use_snc_ssh will be set to true in the ecc_queue record.

[Community Alums]

I was checking logs and saw that our instance has an upgrade that was backed out.

How do I tell if the jar file is available?
        Am I looking for a file named "sncssh.jar" or am I looking for an existing jar file with a particular timestamp? maybe an md5 checksum?

When we set the variable "true" in our development environment, we get REST failures, not SSH failures. - This is confusing to me.

[tim_broberg]

That is confusing. I've never seen that one before.

The jar is "ssh.jar".

SNCSSH is supported starting in Eureka version.

[Community Alums]

we have an ssh.jar, timestamp: 03/10/2015   05:03 AM, filesize: 452,116 bytes, md5sum = 46d11c93c395eac13677b2f7a9f415a5

The REST call that failed does not have the parameters filled in.   It gets some of the parameters from PowerShell, which does return "dir" and "print" statements, but does not return from 'run on remote machine' (from mid-server) when the flag is set.   I am still making and trying tests for this part.

[tim_broberg]

Well, that is confusing because ssh doesn't really interact with Powershell.

You may want to open an incident on this one to sort out the details of what's going on.

    - Tim.

[Community Alums]

Thank you for the responses.   We are opening an incident to find what happened for this particular install.