FIND, GREP and EGREP Commands
FIND, GREP and EGREP are all aliases for the same command. These allow you to search a text file for matches to another string. For example:
HoneyPot /> find NTP jniorsys.log 10/24/17 10:16:38.000, Clock synchronized via NTP (-5) 10/24/17 12:21:59.000, Clock synchronized via NTP (-35) 10/24/17 12:23:21.000, Clock synchronized via NTP (+5) 10/25/17 13:55:55.000, Clock synchronized via NTP (-23) 10/25/17 14:13:59.000, Clock synchronized via NTP (-5) 10/26/17 15:38:34.000, Clock synchronized via NTP (-21) 10/26/17 18:37:50.000, Clock synchronized via NTP (-52) 10/26/17 20:17:50.000, Clock synchronized via NTP (-28) 10/27/17 14:03:21.000, Clock synchronized via NTP (-27) 10/27/17 14:48:22.000, Clock synchronized via NTP (-5) 10/31/17 09:53:57.000, Clock synchronized via NTP (-43) . . .
This can be very useful if you are looking for a specific entry in a log file. Here is the HELP for the command. The only difference in the aliases is that the EGREP usage assumes the -E option allowing the search string to be a standard REGEX expression. It is the same as GREP -E or FIND -E.
HoneyPot /> help egrep EGREP regex filespec Options: regex Regular Expression to match filespec File specification -E Use Regular Expressions (EGREP default) -C Count lines -H Print filespec -N Print line numbers -I Case-independent comparisons -M Underline match Peforms text file search. Aliases: FIND, GREP, EGREP HoneyPot />
The search string (or regex) is case-dependent. You can use the -I option to perform a case-independent search. For eample:
HoneyPot /> find manifest jniorsys.log 11/22/17 08:26:27.567, FTP/50.197.34.73:64227 transferred /flash/manifest.json [116.6 kbps] HoneyPot /> find -i manifest jniorsys.log 11/02/17 08:28:32.943, Manifest updated. 11/02/17 08:32:43.210, Manifest updated. 11/02/17 11:31:03.541, Manifest updated. 11/20/17 07:38:17.390, Manifest updated. 11/22/17 08:26:27.567, FTP/50.197.34.73:64227 transferred /flash/manifest.json [116.6 kbps] HoneyPot />
You will need to enclose you search string in quotes if searching for something containing a space. For example:
HoneyPot /> find -i "manifest updated" jniorsys.log 11/02/17 08:28:32.943, Manifest updated. 11/02/17 08:32:43.210, Manifest updated. 11/02/17 11:31:03.541, Manifest updated. 11/20/17 07:38:17.390, Manifest updated. HoneyPot />
The -C option counts the lines.
HoneyPot /> find -ic "manifest updated" jniorsys.log 6 lines matched HoneyPot />
Alright now there are 6 lines because I just updated using the MANIFEST command twice having seen that it has been a while.
HoneyPot /> find -i "manifest updated" jniorsys.log 11/02/17 08:28:32.943, Manifest updated. 11/02/17 08:32:43.210, Manifest updated. 11/02/17 11:31:03.541, Manifest updated. 11/20/17 07:38:17.390, Manifest updated. 11/22/17 10:20:35.847, Manifest updated. 11/22/17 10:20:45.876, Manifest updated. HoneyPot />
The -N option will include the file line numbers with the matched lines. And, if you know the case of what you are searching for you don’t need the -I option.
HoneyPot /> find -n "Manifest updated" jniorsys.log 356: 11/02/17 08:28:32.943, Manifest updated. 357: 11/02/17 08:32:43.210, Manifest updated. 362: 11/02/17 11:31:03.541, Manifest updated. 812: 11/20/17 07:38:17.390, Manifest updated. 930: 11/22/17 10:20:35.847, Manifest updated. 931: 11/22/17 10:20:45.876, Manifest updated. HoneyPot />
As a general rule (with very few exceptions) options on the command line can be entered separately or combined. The may appear anywhere on the command line.
HoneyPot /> find -i "manifest updated" jniorsys.log -c 6 lines matched HoneyPot />
Commands and options are not case-sensitive by the way.
The -H option not only provides the line number in the file but also includes the file specification which some error processors like to use.
HoneyPot /> find -h "Manifest updated" jniorsys.log /jniorsys.log[356]: 11/02/17 08:28:32.943, Manifest updated. /jniorsys.log[357]: 11/02/17 08:32:43.210, Manifest updated. /jniorsys.log[362]: 11/02/17 11:31:03.541, Manifest updated. /jniorsys.log[812]: 11/20/17 07:38:17.390, Manifest updated. /jniorsys.log[930]: 11/22/17 10:20:35.847, Manifest updated. /jniorsys.log[931]: 11/22/17 10:20:45.876, Manifest updated. HoneyPot />
Note that the output at the command line can be sent to a file using the traditional ‘>’ (create new file) or ‘>>’ (append to file) syntax.
HoneyPot /> find -h "Manifest updated" jniorsys.log > updates.txt HoneyPot /> cat updates.txt /jniorsys.log[356]: 11/02/17 08:28:32.943, Manifest updated. /jniorsys.log[357]: 11/02/17 08:32:43.210, Manifest updated. /jniorsys.log[362]: 11/02/17 11:31:03.541, Manifest updated. /jniorsys.log[812]: 11/20/17 07:38:17.390, Manifest updated. /jniorsys.log[930]: 11/22/17 10:20:35.847, Manifest updated. /jniorsys.log[931]: 11/22/17 10:20:45.876, Manifest updated. HoneyPot />
The Underline match option -M is interesting and unique to JANOS. This is especially useful when debugging a REGEX search. It works for any search as for example:
HoneyPot /> find -m "Manifest updated" jniorsys.log
11/02/17 08:28:32.943, Manifest updated.
----------------
11/02/17 08:32:43.210, Manifest updated.
----------------
11/02/17 11:31:03.541, Manifest updated.
----------------
11/20/17 07:38:17.390, Manifest updated.
----------------
11/22/17 10:20:35.847, Manifest updated.
----------------
11/22/17 10:20:45.876, Manifest updated.
----------------
HoneyPot />
Finally, what about using REGEX? Regular Expressions are a kind of search language which can be used to specify some very complex search requirements. When the -E option or the EGREP command is used the search string can specify a standard REGEX. JANOS implements most but not all aspects of standard Regular Expressions. Here is an example:
HoneyPot /> egrep "NTP \\([+-][0-9][0-9]+)" jniorsys.log 10/24/17 12:21:59.000, Clock synchronized via NTP (-35) 10/25/17 13:55:55.000, Clock synchronized via NTP (-23) 10/26/17 15:38:34.000, Clock synchronized via NTP (-21) 10/26/17 18:37:50.000, Clock synchronized via NTP (-52) 10/26/17 20:17:50.000, Clock synchronized via NTP (-28) 10/27/17 14:03:21.000, Clock synchronized via NTP (-27) 10/31/17 09:53:57.000, Clock synchronized via NTP (-43) 10/31/17 13:03:01.000, Clock synchronized via NTP (-53) 11/01/17 01:37:37.000, Clock synchronized via NTP (-17) 11/02/17 08:28:06.000, Clock synchronized via NTP (-54) 11/07/17 14:27:47.000, Clock synchronized via NTP (-46) 11/14/17 09:07:59.000, Clock synchronized via NTP (-42) 11/15/17 07:59:34.000, Clock synchronized via NTP (-27) 11/17/17 14:01:52.000, Clock synchronized via NTP (-57) 11/21/17 01:37:48.000, Clock synchronized via NTP (-25) 11/21/17 08:57:01.000, Clock synchronized via NTP (-56) 11/22/17 01:33:16.000, Clock synchronized via NTP (-14) HoneyPot /> egrep "NTP \\([+-][0-9])" jniorsys.log -c 14 lines matched HoneyPot />
Here we listed all of the Clock synchronization events that made adjustments of 10 or more milliseconds in either direction. Then we count how many others (less than 10 milliseconds).
It is important to note that when escaping characters in a REGEX on the command line you must escape the escaping character. So in Regex you would escape the open parenthesis (which would normally start a group) using \(. On the command line this must be entered as \\( as you can see.
Try the last search with the -M to underline matches.
HoneyPot /> egrep "NTP \\([+-][0-9])" jniorsys.log -m
10/24/17 10:16:38.000, Clock synchronized via NTP (-5)
--------
\0(8) "NTP (-5)"
10/24/17 12:23:21.000, Clock synchronized via NTP (+5)
--------
\0(8) "NTP (+5)"
10/25/17 14:13:59.000, Clock synchronized via NTP (-5)
--------
\0(8) "NTP (-5)"
10/27/17 14:48:22.000, Clock synchronized via NTP (-5)
.
.
.
HoneyPot />
You can see that with a Regex the matching groups are also displayed along with its length. With Regex, Group 0 (\0) is the entire match. We can separate the magnitude of the adjustment into its own group as follows.
CODE: SELECT ALL
HoneyPot /> egrep "NTP \\([+-]([0-9][0-9]+))" jniorsys.log -m
10/24/17 12:21:59.000, Clock synchronized via NTP (-35)
---------
\0(9) "NTP (-35)"
\1(2) "35"
10/25/17 13:55:55.000, Clock synchronized via NTP (-23)
---------
\0(9) "NTP (-23)"
\1(2) "23"
10/26/17 15:38:34.000, Clock synchronized via NTP (-21)
---------
\0(9) "NTP (-21)"
\1(2) "21"
10/26/17 18:37:50.000, Clock synchronized via NTP (-52)
---------
\0(9) "NTP (-52)"
\1(2) "52"
10/26/17 20:17:50.000, Clock synchronized via NTP (-28)
---------
\0(9) "NTP (-28)"
\1(2) "28"
10/27/17 14:03:21.000, Clock synchronized via NTP (-27)
---------
\0(9) "NTP (-27)"
\1(2) "27"
10/31/17 09:53:57.000, Clock synchronized via NTP (-43)
---------
\0(9) "NTP (-43)"
\1(2) "43"
10/31/17 13:03:01.000, Clock synchronized via NTP (-53)
---------
\0(9) "NTP (-53)"
\1(2) "53"
11/01/17 01:37:37.000, Clock synchronized via NTP (-17)
---------
\0(9) "NTP (-17)"
\1(2) "17"
11/02/17 08:28:06.000, Clock synchronized via NTP (-54)
---------
\0(9) "NTP (-54)"
\1(2) "54"
11/07/17 14:27:47.000, Clock synchronized via NTP (-46)
---------
\0(9) "NTP (-46)"
\1(2) "46"
11/14/17 09:07:59.000, Clock synchronized via NTP (-42)
---------
\0(9) "NTP (-42)"
\1(2) "42"
11/15/17 07:59:34.000, Clock synchronized via NTP (-27)
---------
\0(9) "NTP (-27)"
\1(2) "27"
11/17/17 14:01:52.000, Clock synchronized via NTP (-57)
---------
\0(9) "NTP (-57)"
\1(2) "57"
11/21/17 01:37:48.000, Clock synchronized via NTP (-25)
---------
\0(9) "NTP (-25)"
\1(2) "25"
11/21/17 08:57:01.000, Clock synchronized via NTP (-56)
---------
\0(9) "NTP (-56)"
\1(2) "56"
11/22/17 01:33:16.000, Clock synchronized via NTP (-14)
---------
\0(9) "NTP (-14)"
\1(2) "14"
HoneyPot /> As we hoped Group 1 (\1) contains the part of the match representing the magnitude of the adjustment. You can probably figure out how to include the sign of the adjustment in the group now if that was what you really wanted.
The -M option can be very useful in creating and debugging a Regex for use elsewhere in an application or PHP script.
I should probably cover Regular Expressions in some detail someplace. I put that on the TODO list.
By the way, we have been searching the system log (syslog) as stored in the /jniorsys.log file. That is not all of the event history available. You might notice in the Syslog tab of the DCP that entries go further back in time. So if we had wanted to search all of our syslog history then we could do the following.
HoneyPot /> cat jniorsys.log.bak > syslog.txt HoneyPot /> cat jniorsys.log >> syslog.txt HoneyPot /> egrep "NTP \\([+-][0-9])" syslog.txt 09/27/17 10:40:32.000, Clock synchronized via NTP (+4) 10/24/17 10:16:38.000, Clock synchronized via NTP (-5) 10/24/17 12:23:21.000, Clock synchronized via NTP (+5) 10/25/17 14:13:59.000, Clock synchronized via NTP (-5) 10/27/17 14:48:22.000, Clock synchronized via NTP (-5) 11/14/17 09:23:24.000, Clock synchronized via NTP (+7) 11/14/17 10:08:44.000, Clock synchronized via NTP (-9) 11/15/17 02:13:57.000, Clock synchronized via NTP (+2) 11/15/17 08:06:37.000, Clock synchronized via NTP (+2) 11/15/17 08:11:08.000, Clock synchronized via NTP (+4) 11/15/17 08:15:41.000, Clock synchronized via NTP (+0) 11/15/17 08:18:55.000, Clock synchronized via NTP (-1) 11/15/17 08:21:40.000, Clock synchronized via NTP (+5) 11/17/17 14:45:24.000, Clock synchronized via NTP (-7) 11/18/17 03:26:33.000, Clock synchronized via NTP (-2) HoneyPot /> rm syslog.txt HoneyPot />
Seems that there is only one such event prior to October 24th. The syslog ages to a .BAK file when it grows to about 64KB in size.
This shows how you can concatenate files under JANOS. I haven’t implemented some of the syntax of the DOS COPY command that would let you do that. Maybe some day. There doesn’t seem to be much demand for it.