PIX Parsing (Usable Logs!)

If you have a Cisco PIX you are responsible for, don’t have a logging solution for it, or haven’t seen PLA, keep reading you’re going to be impressed. PDM is out and ASDM is an improvement but who wants to run https-server on a firewall?

There are plenty of commercial solutions available, some just perform log analysis and others attempt to perform event correlation. If you just want to solve the logging problem and are unable to implement a log collection system, take a look at PIX Logging Architecture.

I recently implemented Pix Logging Architecture (PLA) to help manage FWSM, PIX, and ASA logs. I hate having to cull through syslog files looking for traffic stats, tcp port usage, or accepted vs denied traffic information. This point and click world we live in has me spoiled, I don’t want to CLI everything – especially when I’m headed into a meeting to discuss who has access to the Oracle Applications database server.

What if you could get a daily snapshot of your firewall(s)? How about running queries instead of text-based finds? The following log messages are already supportedand reg-ex can be used to further extend the capabilities. Defining traffic types and being able to make graphs? Once you involve a database, you have to decide how often to purge the data.

If you like that and aren’t afraid of using open source, you’ll need apache, MySQL, perl (with some additional modules), syslog daemon or the syslog file, and one more Cisco PIX, ASA, or FWSMs. The OS is up to you, but Linux is highly recommended.

What is PLA?

The PIX Logging Architecture [PLA] is a free and open-source project allowing for correlation of Cisco PIX, Cisco FWSM and Cisco ASA Firewall Traffic, IDS and Informational Logs.

PIX Log message parsing is performed through the use of the PLA parsing module or PLA Msyslogd module. Centralization of the logs is provided using a MySQL database, supported by a Web-based frontend for Log Viewing, Searching, and Event Management. PIX Logging Architecture is completely coded in the Perl programming language, and uses various Perl modules including Perl::DBI and Perl::CGI.

Where is PLA?

PLA’s website has been down since late December but Kris reports it’s BACK ONLINE! If you are interested in downloading the software or documentation, you can find it on SourceForge but your best bet is to go directly to the source itself: PIX Logging Architecture.

I’m working on uploading the documentation and some tweaks to pull in Cisco IPS Event Viewer data into the IDS tab onto my other website, but haven’t been able to reach the original author (Kris) yet.  

How does PLA work?

The PLA software is divided into a parsing piece, database server, and web front-end. Depending on your security policy, network zoning, and server capabilities, you can run this on one server or spread it across several. I was fortunate enough to have access to a dual processor duo2 server, so I implemented everything on the same Linux server.

The underlying architecture is very clever. Message formats are loaded into memory and the syslog file is tailed (via perl module). Messages are flagged by type and processed accordingly, in near-real-time:

The PIX Logging Architecture parsing module, which is responsible for extracting the necessary fields from the PIX system log messages, has been extended to gather new information including, but not limited to, Translations (Xlate’s), Informative Log Messages (i.e. PIX Failover, PIX VPN Establishment, PIX Interface Up/Down, PIX PPPoE VPDN establishment and the like). All the parsing information needed by the PLA Parsing Daemon (pla_parsed) in order to extract data from the logs is now stored in the database, allowing for easy updates of the supported log messages without having to replace the parsing scripts. Moreover, the PLA Parsing Daemon runs as a daemonized Perl process in the background and reads straight and in quasi real-time from the system log files, so no more need to create crontab jobs like before and having to restart syslogd all the time.

The PLA Team has created detailed documentation for installing, tweaking, and supporting PLA. [edit: Google cache of documentation. PLA Documentation can be downloaded at SourceForge].

 In order to get started, you’ll want to read through the first few sections to get a feel for what will be required and figure out what you’ll need in your environment. For example, if you run standard syslogd the parsing of the syslog file works one way and if you run ksyslog or syslog-ng you’ll need to make adjustments. Section 5.5 of the documentation covers tweaking the regex for parsing other syslog engine formats. The documentation explains centralized installation versus distributed, as well as what ports and services will be needed on the server(s).

Security is obviously a concern when processing security log files. Securing the MySQL database access and Apache access is recommended. I would also recommend securing the server itself, if you’re new to Linux this may help.

Enjoy!

Advertisements

Posted on October 22, 2007, in How to's, Security Management and tagged , , , , , . Bookmark the permalink. 15 Comments.

  1. Dear Sirs,
    I have a tremendous problem with PLA.
    I have all the MySQL, Perl and its moduls, pla is running, but it is not putting anything in the DB… I have written to PLA author, but so far no go…
    Can you please point me to some forum, etc?
    PS I really do not agree that the documentation is great. It is quite confusing in places…

  2. PLA Documentation page contains a mailing-list you can subscribe to the support mailing list. Have you emailed: pixla-support-owner@logging-architecture.net

    Looks like their website is having some issues – I’d be happy to help – tack @gmail.com onto the ID I am posting with, to email me directly and I’ll try to help you out.

    -B

  3. Velda – I also ran into the problem of not getting log data into the database after setting everything up. As it turns out, I was using syslog-ng and not syslogd. Apparently syslog-ng cuts one of the date/time stamps that are required by the parsing script. I installed on another machine using syslogd and everything worked as advertised. I am very impressed with this tool.

  4. The online documentation had a section for tweaking the parser’s regex to use other syslog engines (see section 5.5). I’ve added a link to Google’s cached version of the online documentation. I’ve tried emailing Kris to see if hosting the documentation at my other site would be okay, since the main site is down.

  5. Hey guys,

    Im interested in implementing this. But really..im not finding the supported platforms list? Can anyone give me some details as to what OS that used and the versions to get this up and running? jrego@lynn.edu

  6. I like this tool. It’s simple and powerfull.
    But there is a little problem with “pla” and his history management. In my implementation, the daemons receives logs from four ASA firewalls. In three weeks traffic_log table has reached 20 Million records. It’s clear that the amount of log messages is not the problem, but the data model doesn’t take care about performance when making SQL querys on this case.

    I have changed the type for this table (to innodb), including table partitioning (Mysql 5.1) and tweaking some performances parameters, but can’t make it work faster than yesterday.

    It will be great if anybody can share similar experiences.
    Thanks.

  7. Patrux, if you want to share what you’ve already done for changing over the db structure and SQL, I would be glad to take a look at the resulting changes needed in PLA.

    I agree larger record counts will be slower; however, the SELECT statements being run by PLA are single search transactions with nested conditions and unless you’re using this in a multi-user environment it would seem these SELECT statements would fare better in the MyISAM format than InnoDB.

    One could implement some degree of normal-form on the database on IP’s and Port’s but I’m not sure how much that’s going to buy you.

  8. Hi all,

    For information, after 3 months of being offline, the PLA website is back online: http://www.logging-architecture.net.

    Patrux and Bmestep, seen as I am far from an expert on MySQL, I would like to look more closely at the MySQL recommendations to speed up the queries and matching you have done so I can include them in the next version of PLA.

    Thanks,

    Kris

  9. hey!
    that’s cool.

    I’ve spent some time trying to speedup PLA without good results.
    I think that the problem (in my case) is that we’re running “syslog server” and the database on the same system.
    About the database changes, i’ve made changes to storage type on MySQL, adding partitioning and fixes (i hope) to some querys.
    I changed expression:
    log_time>’expr’ and log_time<‘expr’
    for
    log_time between ‘time1’ and ‘time2’
    while i’m using partitions by “range” (to_days) this sentences uses only partitions that contains records in this interval of time. The original sentences uses all partitions.

    Now i’m working to move the database and the front-end to another system.

    I hope that we’re having news about PLA!.

  10. I have a traffic_log database with over 148Million records, I can run some tests on the queries to see which ones perform faster, in the case of the one you mentioned above.

    Look for a new post to capture ideas on enhancing speed / performance of PLA.

  11. Patrux, Bmestep,

    Thanks for the updates. If you want you can contact me via email on kris [at] logging [dash] architecture [dot] net. A traffic_log db of 148 Million records is not bad, how’s that fairing up for you so far? Are you using many display / parse filters? With regards to the speed of the MySQL queries, because of the display filtering being enabled and perfoming a JOIN, it makes the query a little slower. I’ll have to look around to see whether there’s a more effective way of handling these queries.

    Bmestep, I like your write up on using rsyslogd with PLA, do you mind if I put a link to your write up on the PLA website?

    Kris

  12. Hi,
    I change the parameters of the firewall and now I am receiving messagess in my DB…
    Check the configuration of your FW…

  1. Pingback: eliteab » Blog Archive » PIX Parsing (Usable Logs!)

  2. Pingback: Volcaniccep.Com » PIX Parsing (Usable Logs!)

  3. Pingback: BetterTechInfo » Cisco Firewall Logging

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: