Tips and Tricks: Analyzing Tabular Text...

[SlightlyLoony]

It's a fairly common challenge when writing Discovery probes or Runbook activities: you've executed some command on a computer, it returns some text in a tabular format, and now you need to analyze it. For instance, suppose your probe or activity ran on a Linux computer and executed the command

cat /proc/net/arp

You'd get a response that looked something like this, showing that computer's ARP table:

IP address       HW type     Flags       HW address            Mask     Device
10.10.11.101     0x1         0x2         00:0C:29:9D:58:76     *        eth0
10.10.11.102     0x1         0x2         00:26:B0:FA:0E:C7     *        eth0
10.10.10.41      0x1         0x2         00:0C:29:81:1A:49     *        eth0
10.10.10.40      0x1         0x2         00:0C:29:79:B9:56     *        eth0
10.10.10.43      0x1         0x2         00:50:56:AE:7C:F1     *        eth0
10.10.10.70      0x1         0x2         00:26:B9:60:4F:8D     *        eth0
10.10.10.42      0x1         0x2         00:50:56:9C:7E:52     *        eth0

Now suppose that what you really wanted to end up with was a Javascript object with properties named for the IP addresses with values equal to the MAC address, all taken from this data. How would you code that?

Here's some code that will do the job, in a form you can copy and past to System Maintenance → Scripts - Background. The analyze function is the interesting part; the code above that function is just testing it.

var text = "IP address       HW type     Flags       HW address            Mask     Device\n" +
           "10.10.11.101     0x1         0x2         00:0C:29:9D:58:76     *        eth0\n" +
           "10.10.11.102     0x1         0x2         00:26:B0:FA:0E:C7     *        eth0\n" +
           "10.10.10.41      0x1         0x2         00:0C:29:81:1A:49     *        eth0\n" +
           "10.10.10.40      0x1         0x2         00:0C:29:79:B9:56     *        eth0\n" +
           "10.10.10.43      0x1         0x2         00:50:56:AE:7C:F1     *        eth0\n" +
           "10.10.10.70      0x1         0x2         00:26:B9:60:4F:8D     *        eth0\n" +
           "10.10.10.42      0x1         0x2         00:50:56:9C:7E:52     *        eth0\n";

JSUtil.logObject(analyze(text));


function analyze(text) {
   var parser = /^([0-9.]+)\s+0x[0-9a-fA-F]+\s+0x[0-9a-fA-F]+\s+([0-9a-fA-F:]+).*$/gm;
   var result = {};
   var ans;
   while ((ans = parser.exec(text)) != null) {
      var ip = ans[1];
      var mac = ans[2];
      result[ip] = mac;
   }
   return result;
}

All the hard work in this case is being done by the regular expression object (RegExp instance) in the first line of the analyze function. The rest of the function just iterates through all the matches (seven, in this case) within the data and pulls out the IP address and the MAC address. Let's pick that regular expression apart a bit:

/^([0-9.]+)\s+0x[0-9a-fA-F]+\s+0x[0-9a-fA-F]+\s+([0-9a-fA-F:]+).*$/gm

What is all this stuff?

• The slashes ("/") just mark the beginning and the end of the regular expression.
• The aqua "gm" at the end are flags that tell the regular expression to be global and multi-line. The global flag tells the regular expression that it's looking for more than one match (which we need, because we want to match each IP/MAC pair). The multi-line flag tells the regular expression that it's looking for matches on more than one line.
• The black "^" and "$" mean beginning of line and end of line, respectively. In other words, we're going to match an entire line with all the stuff between the ^ and the $.
• The red bits in in parentheses are capture groups; they define the text we actually want to look at. The first one is looking for one or more digits or a periods in a row — the IP address. The second one is looking for one or more hexadecimal digits or a colon in a row — the MAC address.
• The blue "\s+" sequences match one or more blank spaces (aka "whitespace") — the areas between the columns of data.
• The pink ".*" matches anything at all, including nothing.
• Finally, the green sequences match the hexadecimal number in the HW Type and Flags columns.

Put all this together and that regular expression matches entire lines that consist exactly (and only) of the following (in order):

• The beginning of the line.
• An IP address (which we capture).
• Some whitespace.
• A hexadecimal number.
• Some whitespace.
• A hexadecimal number.
• Some whitespace.
• A MAC address (which we capture).
• Anything at all.
• The end of the line.

Once we match a line, all we have to do is read the capture groups (1 and 2) from the array returned after calling the exec() function.

Oh, and if you're not all that familiar with sea creatures and you're wondering what's going on in that photo — it's a fisherman who just caught a squid, and the unhappy squid has just squirted his "ink" in an effort to escape. The fisherman probably wasn't too happy about this, either...