Tweaking PLA: Using rsyslog
Posted by bmestep on March 11, 2008
[BetterTechInfo has an thorough PLA Syslog Configuration article now]
PLA (PIX Logging Architecture) uses regular expressions (regex) to parse syslog messages received from Cisco firewallsand comes pre-configured to process standard “syslogd” message format. Most current Linux distributions ship with rsyslog (able to log directly to MySQL) while some administrators prefer syslog-ng.
The installation documentation distributed with PLA assumes a familiarity regex, so here you’ll see how to tweak PLA to parse your rsyslogd log file.
Perl is used to parse through the syslog message looking for matches to message formats described in the syslog_messages table in the pix database. The processing script pla_parsedcontains a regex pattern that must be matched in order for the processing to occur. The applicable section is:
### PIX Firewall Logs
### DEFINE YOUR SYSLOG LAYOUT HERE!
###$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) (.*)“;
$var_pixhost=3;
$var_pixmonth=4;
$var_pixdate=5;
$var_pixyear=6;
$var_pixtime=7;
Here, the variable regex_log_beginneeds to match up all the log information up to the PIX, ASA, or FWSM message code in order to understand date, time, and host for these messages. Take a look at the provided sample log entry, everything in red needs to be picked up by regex_log_begin while the remainder is standard for Cisco firewalls:
Oct 21 23:59:23 fwext-dmz-01 Oct 21 2006 23:58:23: %PIX-6-305011: Built dynamic TCP translation from inside:1.1.1.1/2244 to outside:2.2.2.2/3387
Explaining the operation of regex and wildcards is beyond the scope of this article; however, numerous guides have been written to fill the void. In our case, adjusting the default regex to match rsyslog is straight forward after noting which characters match which pattern, again we’re working with the basics of regex here - nothing fancy.
Take this sample rsyslog entry and notice the difference from the standard syslogd format:
Feb 21 10:59:32 Feb 21 2008 10:59:32 pix4us : %PIX-6-110001: No route to 1.1.1.1 from 3.4.5.6
Feb 21 10:59:32 Feb 21 2008 10:59:32 pix4us
Oct 21 23:59:23 fwext-dmz-01 Oct 21 2006 23:58:23
Here, the rsyslog entry includes the date twice and then the hostname of the log source versus the default format expected by pla_parsedof date hostname date. The original regex is set to pickup the first time entry’s “minutes and seconds” and picks up the next 5 words/entries separated by spaces:
$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) (.*)“;
Oct 21 23:59:23 fwext-dmz-01 Oct 21 2006 23:58:23
In order to process rsyslog, this will have to be changed. The initial (.*):(.*) is used to set a starting point in syslog message string. Since this new rsyslog format includes two date entries before the host name, the following can be used to allow pla_parsedto “see” the new syslog message string:
$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)“;
Feb 21 10:59:32 Feb 21 2008 10:59:32 pix4us
The regex starts out the same, but looking at the colors you will notice the location of the information needed by pla_parsed to determine date, time, and host has moved. This time we used “(.*):(.*)” and “((.*):(.*):(.*))” to force a match on the time elements.
As a result of this change, the variables listed below the regex pattern must be modified to tell pla_parsed which (.*) contains which element:
$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)“;
$var_pixhost=7;
$var_pixmonth=3;
$var_pixdate=4;
$var_pixyear=5;
$var_pixtime=6;
The numbering happens left to right and the color coding should help this make sense. The ()’s around the grey time entry are grouped together and count as one match/entity, the sixth variable. This same approach of keying off the timestamping can be applied to pla_parsedin order to allow processing of syslog-ng, ksyslogd, or any other syslog message format.
Need help with a different format? Have problems getting your PIX logs loaded? Paste in a sample message from your syslog server (IP Addresses santized please) in a comment below.
March 18, 2008 at 11:48 am
[...] Tweaking PLA: Using rsyslog [...]
March 19, 2008 at 11:35 pm
using syslog-ng, I get the following format for pix log entries:
2008-03-19T22:32:46-07:00 1.1.1.5 Mar 19 2008 21:32:46: %PIX-4-106023: Deny tcp src outside:58.224.81.118/3603 dst pridmz:server01/25 by access-group “outside”
would the following be a suitable regex for this?
$regex_log_begin = “(.*) (.*) (.*) (.*) (.*) (.*)”
$var_pixhost=2;
$var_pixmonth=3;
$var_pixdate=4;
$var_pixyear=5;
$var_pixtime=6;
please let me know, thx!!
March 20, 2008 at 11:41 am
It would be more reliable to start with the original regex “(.*):(.*)…” and then map the variables as follows:
$var_pixhost=3;
$var_pixmonth=4;
$var_pixdate=5;
$var_pixyear=6;
$var_pixtime=7;
I suspect the initial timestamp is throwing the regex off.
You can get regex to better match your log format by replacing the 7th (.*) with the format I used in the rsyslog example above: ((.*):(.*):(.*))
Drop me an email, if that doesn’t work or if you need help translating that into the pla_parsed file. bmestep (is_at) gmail (dot) com
March 20, 2008 at 12:22 pm
Chris,
I made a quick perl hack to find the solution and I’ll throw together a post on it shortly, in the meantime…
Stick a ; at the end of your regex you pasted above and you should be set.
Change your variables to this:
$var_pixhost=2;
$var_pixmonth=3;
$var_pixdate=4;
$var_pixyear=5;
$var_pixtime=6;
The original regex should have worked, based on testing the message you provided and the $regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) ((.*):(.*):(.*))”; will definitely work.
May 22, 2008 at 3:51 pm
Chris,
The following is what my logs look like now:
2008 05 22 11:36:35 192.168.0.1 05 22 2008 11:36:35 %PIX-6-305012: Teardown dynamic TCP translation from inside:192.168.0.48/2457 to outside:1.1.1.1/57097 duration 0:00:31
The following is from my pla_parsed script:
$regex_log_begin = “(.*) (.*) (.*) (.*) (.*) (.*)”;
#$regex_lob_begin = “(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)”;
$var_pixhost=2;
$var_pixmonth=3;
$var_pixdate=4;
$var_pixyear=5;
$var_pixtime=6;
I am getting nothing in my DB at all and am not sure if my regex_lob_begin is correct.
Thanks.