PIX Logging Architecture is Back Online

Great news, after a brief recess, PIX Logging Architecture is back on the NET!

Be sure to checkout the screenshots for features / selling points and import the latest syslog-message database if you’ve not done that since installation. Remember PLA handles the following Cisco Security Devices:

PIX Logging Architecture v2.00 supports log messages from the following devices:

• Cisco ASA (TESTED AND CONFIRMED)
• Cisco PIX v6.x (TESTED AND CONFIRMED)
• Cisco PIX v7.x (TESTED AND CONFIRMED)
• Cisco FWSM 2.x (TESTED AND CONFIRMED)
• Cisco FWSM 3.x (TESTED AND CONFIRMED)

PLA Documentation

PLA Screenshots and more PLA Screenshots

PLA: Latest PLA syslog message support

Support for alternate forms of syslog daemons can also be found here for parsing rsyslog.

Welcome back Kris!

Tweaking PLA: Using rsyslog

[BetterTechInfo has an thorough PLA Syslog Configuration article now]

PLA (PIX Logging Architecture) uses regular expressions (regex) to parse syslog messages received from Cisco firewallsand comes pre-configured to process standard “syslogd” message format. Most current Linux distributions ship with rsyslog (able to log directly to MySQL) while some administrators prefer syslog-ng.

The installation documentation distributed with PLA assumes a familiarity regex, so here you’ll see how to tweak PLA to parse your rsyslogd log file.

Perl is used to parse through the syslog message looking for matches to message formats described in the syslog_messages table in the pix database. The processing script pla_parsedcontains a regex pattern that must be matched in order for the processing to occur. The applicable section is:

### PIX Firewall Logs
### DEFINE YOUR SYSLOG LAYOUT HERE!
###$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) (.*)“;
$var_pixhost=3;
$var_pixmonth=4;
$var_pixdate=5;
$var_pixyear=6;
$var_pixtime=7;

Here, the variable regex_log_beginneeds to match up all the log information up to the PIX, ASA, or FWSM message code in order to understand date, time, and host for these messages. Take a look at the provided sample log entry, everything in red needs to be picked up by regex_log_begin while the remainder is standard for Cisco firewalls:

Oct 21 23:59:23 fwext-dmz-01 Oct 21 2006 23:58:23: %PIX-6-305011: Built dynamic TCP translation from inside:1.1.1.1/2244 to outside:2.2.2.2/3387

Explaining the operation of regex and wildcards is beyond the scope of this article; however, numerous guides have been written to fill the void. In our case, adjusting the default regex to match rsyslog is straight forward after noting which characters match which pattern, again we’re working with the basics of regex here - nothing fancy.

Take this sample rsyslog entry and notice the difference from the standard syslogd format:

Feb 21 10:59:32 Feb 21 2008 10:59:32 pix4us : %PIX-6-110001: No route to 1.1.1.1 from 3.4.5.6

Feb 21 10:59:32 Feb 21 2008 10:59:32 pix4us
Oct 21 23:59:23 fwext-dmz-01 Oct 21 2006 23:58:23

Here, the rsyslog entry includes the date twice and then the hostname of the log source versus the default format expected by pla_parsedof date hostname date. The original regex is set to pickup the first time entry’s “minutes and seconds” and picks up the next 5 words/entries separated by spaces:

$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) (.*)“;

 Oct 21 23:59:23 fwext-dmz-01 Oct 21 2006 23:58:23

In order to process rsyslog, this will have to be changed. The initial (.*):(.*) is used to set a starting point in syslog message string. Since this new rsyslog format includes two date entries before the host name, the following can be used to allow pla_parsedto “see” the new syslog message string:

$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)“;

Feb 21 10:59:32 Feb 21 2008 10:59:32 pix4us

The regex starts out the same, but looking at the colors you will notice the location of the information needed by pla_parsed to determine date, time, and host has moved. This time we used “(.*):(.*)” and “((.*):(.*):(.*))” to force a match on the time elements.

As a result of this change, the variables listed below the regex pattern must be modified to tell pla_parsed which (.*) contains which element:

$regex_log_begin = “(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)“;

$var_pixhost=7;
$var_pixmonth=3;
$var_pixdate=4;
$var_pixyear=5;
$var_pixtime=6;

The numbering happens left to right and the color coding should help this make sense. The ()’s around the grey time entry are grouped together and count as one match/entity, the sixth variable. This same approach of keying off the timestamping can be applied to pla_parsedin order to allow processing of syslog-ng, ksyslogd, or any other syslog message format.

Need help with a different format? Have problems getting your PIX logs loaded? Paste in a sample message from your syslog server (IP Addresses santized please) in a comment below.

PIX Parsing (Usable Logs!)

If you have a Cisco PIX you are responsible for, don’t have a logging solution for it, or haven’t seen PLA, keep reading you’re going to be impressed. PDM is out and ASDM is an improvement but who wants to run https-server on a firewall?

There are plenty of commercial solutions available, some just perform log analysis and others attempt to perform event correlation. If you just want to solve the logging problem and are unable to implement a log collection system, take a look at PIX Logging Architecture.

I recently implemented Pix Logging Architecture (PLA) to help manage FWSM, PIX, and ASA logs. I hate having to cull through syslog files looking for traffic stats, tcp port usage, or accepted vs denied traffic information. This point and click world we live in has me spoiled, I don’t want to CLI everything - especially when I’m headed into a meeting to discuss who has access to the Oracle Applications database server.

What if you could get a daily snapshot of your firewall(s)? How about running queries instead of text-based finds? The following log messages are already supportedand reg-ex can be used to further extend the capabilities. Defining traffic types and being able to make graphs? Once you involve a database, you have to decide how often to purge the data.

If you like that and aren’t afraid of using open source, you’ll need apache, MySQL, perl (with some additional modules), syslog daemon or the syslog file, and one more Cisco PIX, ASA, or FWSMs. The OS is up to you, but Linux is highly recommended.

What is PLA?

The PIX Logging Architecture [PLA] is a free and open-source project allowing for correlation of Cisco PIX, Cisco FWSM and Cisco ASA Firewall Traffic, IDS and Informational Logs.

PIX Log message parsing is performed through the use of the PLA parsing module or PLA Msyslogd module. Centralization of the logs is provided using a MySQL database, supported by a Web-based frontend for Log Viewing, Searching, and Event Management. PIX Logging Architecture is completely coded in the Perl programming language, and uses various Perl modules including Perl::DBI and Perl::CGI.

Where is PLA?

PLA’s website has been down since late December but Kris reports it’s BACK ONLINE! If you are interested in downloading the software or documentation, you can find it on SourceForge but your best bet is to go directly to the source itself: PIX Logging Architecture.

I’m working on uploading the documentation and some tweaks to pull in Cisco IPS Event Viewer data into the IDS tab onto my other website, but haven’t been able to reach the original author (Kris) yet.  

How does PLA work?

The PLA software is divided into a parsing piece, database server, and web front-end. Depending on your security policy, network zoning, and server capabilities, you can run this on one server or spread it across several. I was fortunate enough to have access to a dual processor duo2 server, so I implemented everything on the same Linux server.

The underlying architecture is very clever. Message formats are loaded into memory and the syslog file is tailed (via perl module). Messages are flagged by type and processed accordingly, in near-real-time:

The PIX Logging Architecture parsing module, which is responsible for extracting the necessary fields from the PIX system log messages, has been extended to gather new information including, but not limited to, Translations (Xlate’s), Informative Log Messages (i.e. PIX Failover, PIX VPN Establishment, PIX Interface Up/Down, PIX PPPoE VPDN establishment and the like). All the parsing information needed by the PLA Parsing Daemon (pla_parsed) in order to extract data from the logs is now stored in the database, allowing for easy updates of the supported log messages without having to replace the parsing scripts. Moreover, the PLA Parsing Daemon runs as a daemonized Perl process in the background and reads straight and in quasi real-time from the system log files, so no more need to create crontab jobs like before and having to restart syslogd all the time.

The PLA Team has created detailed documentation for installing, tweaking, and supporting PLA. [edit: Google cache of documentation. PLA Documentation can be downloaded at SourceForge].

 In order to get started, you’ll want to read through the first few sections to get a feel for what will be required and figure out what you’ll need in your environment. For example, if you run standard syslogd the parsing of the syslog file works one way and if you run ksyslog or syslog-ng you’ll need to make adjustments. Section 5.5 of the documentation covers tweaking the regex for parsing other syslog engine formats. The documentation explains centralized installation versus distributed, as well as what ports and services will be needed on the server(s).

Security is obviously a concern when processing security log files. Securing the MySQL database access and Apache access is recommended. I would also recommend securing the server itself, if you’re new to Linux this may help.

Enjoy!

Log Management 101

If you are in any way involved with technology, you have at one time or another dealt with logging. You either didn’t have what you needed, spent hours looking a needle in the haystack, or had to go find someone to help you get what you needed.

In networking and security circles, logging is usually a hot-button topic. I’ve spent many man-months working with logs, investigating logging solutions, praying to God for a solution to logging, tweaking open-source logging programs, and discussing logging technology with various vendors. I’ve collected some of the more relevant articles on the topic and have some additional commentary / insight.

First and foremost, the primary problem of Log Management is often not the log(ging) portion - most systems spew limitless log data - the problem is the management of the log(ged) data. You’ve heard 1000 times, management styles vary by manager. Management of network, servers, and security differ wildly. Networks and servers are managed by exception, they get attention when they need it otherwise they just purr and no one bothers them (until Patch Tuesday or until Cisco has a bad day). Managing security is altogether different, because security for a given organization is the net sum of all the parts of said organization, and in order to properly manage security, information from all the parts must be reviewed on a routine basis.

This routine review of information is where the problem comes in and it is why nearly every regulation, guideline, or contract stipulation (read PCI DSS) has mandatory language about reviewing logs. Again there is seldom anything about requiring systems to output logs, because that is rarely the problem. Technology is often times misapplied in attempts solve human-created problems, log management is an ideal instance for technology to come to our rescue. How else would a sane person cull through gigabytes or terabytes of log data from ten’s or hundred’s of systems or network elements? <Don’t answer that question, I’ve done it myself .>

Logging resources

• Anton Chuvakin’s Security Warrior Blog
• Network Systems Design Line: Security Event Correlation Article
• GEEK Speek Blog: Dealing with Logs
• TaoSecurity: Log Mgmt Summit
• SANS Reading Room: Logging Technology / Techniques

Dealing with the actual logs and delving into whether you want to collect log data, consolidate the log data, correlate log data, and or perform event management usher in the next set of issues / challenges when dealing with logging. Originally logs might have been aggregated onto one server or they might have been aggregated by log severity using syslog-ng or syslogd. Network or systems management systems (CiscoWorks or HP OpenView) might have received logs and performed parsing or filtering on key events that drove alerts and alarms but these systems likely discarded the log data itself, so investigating the root cause of an alert or incident became next to impossible.

Compliance came to the scene and brought innovation in log collection and log management; software became available to manage logs from multiple sources but you had to pick your poison: Windows with MS SQL back-end, Solaris or HP-UX with Oracle, or Linux and MySQL. Aggregation and consolidation were solved by tiered deployments and agents were usually installed to siphon off Windows Event Data. Security hit the mainstream and event correlation was the big buzz word.

The problem was how the intensive resources necessary to store the log data competed with the intensive resources necessary to correlate dissimilar log formats to produce alerts or suppress logged events. Gartner and others coined the the term SIEM, which combined log collection and event correlation, and a new market arrived. Most SIEM (SIM, then SEM, finally SIEM) are really good at the collection piece (historical) or really good at the correlation (real-time) piece, while few are good at both. Go, go Magic Quad. rant! If you hadn’t already noticed, I’m not a fan of combining these technologies (I don’t mix my food while I’m eating either). I like my logging solution to have plenty of evidence preservation goodness and I don’t want it muddied because a correlator had to normalize the data before it could parse, alarm on, or display the log data.

Some of the options for solving log management challenges

• LogRhythm
• LogLogic
• ArcSight
• Splunk
• Got a PIX?
• Yummy Kiwi
• Collection of Log options

Just scratching the surface, more please?