Tweaking PLA: Using rsyslog

PLA (PIX Logging Architecture) uses regular expressions (regex) to parse syslog messages received from Cisco firewallsand comes pre-configured to process standard “syslogd” message format. Most current Linux distributions ship with rsyslog (able to log directly to MySQL) while some administrators prefer syslog-ng.

The installation documentation distributed with PLA assumes a familiarity regex, so here you’ll see how to tweak PLA to parse your rsyslogd log file.

Perl is used to parse through the syslog message looking for matches to message formats described in the syslog_messages table in the pix database. The processing script pla_parsedcontains a regex pattern that must be matched in order for the processing to occur. The applicable section is:

### PIX Firewall Logs
### DEFINE YOUR SYSLOG LAYOUT HERE!
###
$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) (.*)“;
$var_pixhost=3;
$var_pixmonth=4;
$var_pixdate=5;
$var_pixyear=6;
$var_pixtime=7;

Here, the variable regex_log_beginneeds to match up all the log information up to the PIX, ASA, or FWSM message code in order to understand date, time, and host for these messages. Take a look at the provided sample log entry, everything in red needs to be picked up by regex_log_begin while the remainder is standard for Cisco firewalls:

Oct 21 23:59:23 fwext-dmz-01 Oct 21 2006 23:58:23: %PIX-6-305011: Built dynamic TCP translation from inside:1.1.1.1/2244 to outside:2.2.2.2/3387

Explaining the operation of regex and wildcards is beyond the scope of this article; however, numerous guides have been written to fill the void. In our case, adjusting the default regex to match rsyslog is straight forward after noting which characters match which pattern, again we’re working with the basics of regex here – nothing fancy.

Take this sample rsyslog entry and notice the difference from the standard syslogd format:

Feb 21 10:59:32 Feb 21 2008 10:59:32 pix4us : %PIX-6-110001: No route to 1.1.1.1 from 3.4.5.6

Feb 21 10:59:32 Feb 21 2008 10:59:32 pix4us
Oct 21 23:59:23 fwext-dmz-01 Oct 21 2006 23:58:23

Here, the rsyslog entry includes the date twice and then the hostname of the log source versus the default format expected by pla_parsedof date hostname date. The original regex is set to pickup the first time entry’s “minutes and seconds” and picks up the next 5 words/entries separated by spaces:

$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) (.*)“;

 Oct 21 23:59:23 fwext-dmz-01 Oct 21 2006 23:58:23

In order to process rsyslog, this will have to be changed. The initial (.*):(.*) is used to set a starting point in syslog message string. Since this new rsyslog format includes two date entries before the host name, the following can be used to allow pla_parsedto “see” the new syslog message string:

$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)“;

Feb 21 10:59:32 Feb 21 2008 10:59:32 pix4us

The regex starts out the same, but looking at the colors you will notice the location of the information needed by pla_parsed to determine date, time, and host has moved. This time we used “(.*):(.*)” and “((.*):(.*):(.*))” to force a match on the time elements.

As a result of this change, the variables listed below the regex pattern must be modified to tell pla_parsed which (.*) contains which element:

$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)“;

$var_pixhost=7;
$var_pixmonth=3;
$var_pixdate=4;
$var_pixyear=5;
$var_pixtime=6;

The numbering happens left to right and the color coding should help this make sense. The ()’s around the grey time entry are grouped together and count as one match/entity, the sixth variable. This same approach of keying off the timestamping can be applied to pla_parsedin order to allow processing of syslog-ng, ksyslogd, or any other syslog message format.

Need help with a different format? Have problems getting your PIX logs loaded? Paste in a sample message from your syslog server (IP Addresses santized please) in a comment below.

Advertisements

Posted on March 11, 2008, in How to's, Security Management and tagged , , , , , , . Bookmark the permalink. 18 Comments.

  1. using syslog-ng, I get the following format for pix log entries:

    2008-03-19T22:32:46-07:00 1.1.1.5 Mar 19 2008 21:32:46: %PIX-4-106023: Deny tcp src outside:58.224.81.118/3603 dst pridmz:server01/25 by access-group “outside”

    would the following be a suitable regex for this?

    $regex_log_begin = “(.*) (.*) (.*) (.*) (.*) (.*)”
    $var_pixhost=2;
    $var_pixmonth=3;
    $var_pixdate=4;
    $var_pixyear=5;
    $var_pixtime=6;

    please let me know, thx!!

  2. It would be more reliable to start with the original regex “(.*):(.*)…” and then map the variables as follows:

    $var_pixhost=3;
    $var_pixmonth=4;
    $var_pixdate=5;
    $var_pixyear=6;
    $var_pixtime=7;

    I suspect the initial timestamp is throwing the regex off.
    You can get regex to better match your log format by replacing the 7th (.*) with the format I used in the rsyslog example above: ((.*):(.*):(.*))

    Drop me an email, if that doesn’t work or if you need help translating that into the pla_parsed file. bmestep (is_at) gmail (dot) com

  3. Chris,

    I made a quick perl hack to find the solution and I’ll throw together a post on it shortly, in the meantime…

    Stick a ; at the end of your regex you pasted above and you should be set.
    Change your variables to this:

    $var_pixhost=2;
    $var_pixmonth=3;
    $var_pixdate=4;
    $var_pixyear=5;
    $var_pixtime=6;

    The original regex should have worked, based on testing the message you provided and the $regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) ((.*):(.*):(.*))”; will definitely work.

  4. Chris,

    The following is what my logs look like now:

    2008 05 22 11:36:35 192.168.0.1 05 22 2008 11:36:35 %PIX-6-305012: Teardown dynamic TCP translation from inside:192.168.0.48/2457 to outside:1.1.1.1/57097 duration 0:00:31

    The following is from my pla_parsed script:

    $regex_log_begin = “(.*) (.*) (.*) (.*) (.*) (.*)”;
    #$regex_lob_begin = “(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)”;
    $var_pixhost=2;
    $var_pixmonth=3;
    $var_pixdate=4;
    $var_pixyear=5;
    $var_pixtime=6;

    I am getting nothing in my DB at all and am not sure if my regex_lob_begin is correct.

    Thanks.

  5. Ok this has been driving me crazy. Here is the config snip:
    $regex_log_begin =”(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)”;
    $var_pixhost=9;
    $var_pixmonth=3;
    $var_pixdate=4;
    $var_pixyear=5;
    $var_pixtime=6;

    It is rsyslog on a fedora box. Here is the log header:

    Oct 23 13:53:18 Oct 23 2008 13:53:18 CTXDFTI0000O02

    If I use what you have I get nothing using what I have here gets me entries, almost correct. But it is parsing the Firewall name wrong, it is rolling the last part of the time into it.

    Here is the database entry:

    13934160 2008-10-23 13:50:45 45 CTXDFTI0000O02 ASA-7-710005 DROP UDP 0.0.0.0 68 255.255.255.255 6

    It thinks that “45 CTXDFTI0000O02” is the log source so I wind up with 60*number of devices reporting to the server.

    Anyone?

  6. fuente:~ # perl pla_parser_regex_test.txt

    You supplied the following log message:

    11 03 18:22:05 ciscoasa 11 03 2008 18:22:05

    Resulting in these matches using a default regex filter of:

    (.*):(.*) (.*) (.*) (.*) (.*) (.*)
    _________________________________________________________

    ]> match 1: 11 03 18:22
    ]> match 2: 05
    ]> match 3: ciscoasa (default pixhost)
    ]> match 4: 11 (default pixmonth)
    ]> match 5: 03 (default pixdate)
    ]> match 6: 2008 (default pixyear)
    ]> match 7: 18:22:05 (default pixtime)
    _________________________________________________________

  7. My rsyslog entry is :

    Jan 25 10:39:55 Jan 25 2009 10:26:42: %ASA-4-106023: Deny udp src FSL_White_Zone:JAL_PLA_SRVR/57740 dst FSL_IPLC_Zone:192.168.81.168/53 by access-group “FSL_White_Zone_access_in” [0x0, 0x0]

    I am stuggling to find the correct regrex entry for:
    Jan 25 10:39:55 Jan 25 2009 10:26:42

    Plz help!!

    • Naveen,

      Your log stream doesn’t include the hostname of the logging source (firewall). If this is by design, you’ll want to hardcode the ‘pixhost’ to the name you want to use.

      You’ll can try chaning your regex to “(.*):(.*) (.*) (.*) (.*) (.*:?:?)”
      Which should translate into matches:

      matched: Jan 25 10:39:55 Jan 25 2009 10:26:42
      ]> match 3: Jan (default pixhost)
      ]> match 4: 25 (default pixmonth)
      ]> match 5: 2009 (default pixdate)
      ]> match 6: 10:26:42 (default pixyear)
      ]> match 7: (default pixtime)

      I’m not a reg-ex geek, but that should work. Let us know how you fare.

      – Brian

  8. Hello,
    my Problem the PLA Log will not work with date format if the day is one digit, but it work perfect with 2 digits

    will not match
    Feb 3 12:15:23 asa 2009 Feb 3 12:15:23
    it’s matched
    Dec 16 10:18:37 asa 2008 Dec 16 10:18:37

    $regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) (.*)”;
    $var_pixhost=3;
    $var_pixmonth=5;
    $var_pixdate=6;
    $var_pixyear=4;
    $var_pixtime=7;

    How can I change the regex to work with both?

    • RM,

      It looks like your ASA is inserting an extra space for single digit dates. The regex supplied matches on any character, including extra spaces. I tried converting the regex to other patterns to find one that might help you out.
      Try this and see if it helps: (.*):(.*) (.*) (.*) (.*) (.*) (.*:?:?)
      It should force a match more consistently.

      Let us know how it goes.

      — Brian

  9. Hello,

    the result is:
    You supplied the following log message:
    Feb 5 14:08:45 de9cf000 2009 Feb 5 14:08:44

    Resulting in these matches using a default regex filter of:
    (.*):(.*) (.*) (.*) (.*) (.*) (.*:?:?)
    _________________________________________________________

    ]> match 1: Feb 5 14:08
    ]> match 2: 45 de9cf000
    ]> match 3: 2009 (default pixhost)
    ]> match 5: (default pixmonth)
    ]> match 6: 5 (default pixdate)
    ]> match 4: Feb (default pixyear)
    ]> match 7: 14:08:44 (default pixtime)
    _________________________________________________________

    I changed in syslog-ng the datetime to $S_ISODATE and it works
    You supplied the following log message:
    Feb 5 14:39:30 de9cf000 2009-02-05T14:39:30+0100

    Resulting in these matches using a default regex filter of:
    (.*):(.*) (.*) (.*)-(.*)-(.*)T(.*)[+-]
    _________________________________________________________

    ]> match 1: Feb 5 14:39
    ]> match 2: 30
    ]> match 3: de9cf000 (default pixhost)
    ]> match 5: 02 (default pixmonth)
    ]> match 6: 05 (default pixdate)
    ]> match 4: 2009 (default pixyear)
    ]> match 7: 14:39:30 (default pixtime)
    _________________________________________________________

    but I must also change ## Calculates correct date
    %months = ( from “Jan”,”01″, to “01”,”01″, etc.

    thank you for support.

  10. Problem: single digit date with leading whitespace.

    Supplied log message:
    Feb 9 07:57:22 INTP-SAC Feb 9 07:57:22 2010

    currently using (in pla_parsed):

    $regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) (.*)”;
    $var_pixhost=3;
    $var_pixmonth=4;
    $var_pixdate=5;
    $var_pixyear=7;
    $var_pixtime=6;

    ——–

    Works fine after the 10th of each month.

    • To my above post. It seems that even wordpress parses out the double whitespace in the single digit date.

      in the log, it is Feb9 … two shitespaces between month and date.

      Thank you for any insight you can offer.

      gtch

  11. Hi,
    we are using ASA firewall and log entry looks like :
    20 04:16:35 x.x.x.x Sep 20 2012 04:16:32: %ASA-6-305012: Teardown dynamic UDP translation from inside:10.237.3.31/55467 to outside:DNBTEST/26550 duration 0:02:41

    please suggest us what should be the regex entry.We are not getting logs entry in mysql database

  12. You post interesting articles here. Your page deserves much
    more visitors. It can go viral if you give it initial boost, i know useful tool that can help you,
    simply type in google: svetsern traffic tips

  13. Hi, I working one my configuration for 2 day and i stuck. I hope you can give some advice.

    My rsyslog line on Centos7 is

    Feb 10 16:50:56 192.168.13.254 2016 : Feb 10 14:32:48 UTC: %ASA-session-6-305012: Teardown dynamic TCP translation from inside:192.168.13.3/45421 to outside:192.168.11.253/45421 duration 0:00:30.

    I had t0 reconfigure my asa configuration to set a second Month, Date end Time entry.

    Until this moment i have;

    $regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)”;
    $var_pixhost=3;
    $var_pixyear=4;
    $var_pixmonth=6;
    $var_pixdate=7;
    $var_pixtime=8;

    match 1 =Feb 10 16:50
    match 2=:56
    match 3=192.168.13.254
    match 4=2016
    match 5=:
    match 6=Feb
    match 7=10
    match 8=14:32:48
    match 9=UTC:
    Rest = %ASA-session-6-305012: Teardown dynamic TCP translation from inside:192.168.13.3/45421 to outside:192.168.11.253/45421 duration 0:00:30.

    Best regards,

    Pas

  1. Pingback: PIX Logging Architecture is Back Online « Practical Tactics

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: