The following is what my logs look like now:

2008 05 22 11:36:35 192.168.0.1 05 22 2008 11:36:35 %PIX-6-305012: Teardown dynamic TCP translation from inside:192.168.0.48/2457 to outside:1.1.1.1/57097 duration 0:00:31

The following is from my pla_parsed script:

$regex_log_begin = “(.*) (.*) (.*) (.*) (.*) (.*)”;
#$regex_lob_begin = “(.*):(.*) (.*) (.*) (.*) ((.*):(.*):(.*)) (.*)”;
$var_pixhost=2;
$var_pixmonth=3;
$var_pixdate=4;
$var_pixyear=5;
$var_pixtime=6;

I am getting nothing in my DB at all and am not sure if my regex_lob_begin is correct.

Thanks.

- Sue.

I made a quick perl hack to find the solution and I’ll throw together a post on it shortly, in the meantime…

Stick a ; at the end of your regex you pasted above and you should be set.
Change your variables to this:

$var_pixhost=2;
$var_pixmonth=3;
$var_pixdate=4;
$var_pixyear=5;
$var_pixtime=6;

The original regex should have worked, based on testing the message you provided and the $regex_log_begin = “(.*):(.*) (.*) (.*) (.*) (.*) ((.*):(.*):(.*))”; will definitely work.

$var_pixhost=3;
$var_pixmonth=4;
$var_pixdate=5;
$var_pixyear=6;
$var_pixtime=7;

I suspect the initial timestamp is throwing the regex off.
You can get regex to better match your log format by replacing the 7th (.*) with the format I used in the rsyslog example above: ((.*):(.*):(.*))

Drop me an email, if that doesn’t work or if you need help translating that into the pla_parsed file. bmestep (is_at) gmail (dot) com

2008-03-19T22:32:46-07:00 1.1.1.5 Mar 19 2008 21:32:46: %PIX-4-106023: Deny tcp src outside:58.224.81.118/3603 dst pridmz:server01/25 by access-group “outside”

would the following be a suitable regex for this?

$regex_log_begin = “(.*) (.*) (.*) (.*) (.*) (.*)”
$var_pixhost=2;
$var_pixmonth=3;
$var_pixdate=4;
$var_pixyear=5;
$var_pixtime=6;

please let me know, thx!!

Thanks for the updates. If you want you can contact me via email on kris [at] logging [dash] architecture [dot] net. A traffic_log db of 148 Million records is not bad, how’s that fairing up for you so far? Are you using many display / parse filters? With regards to the speed of the MySQL queries, because of the display filtering being enabled and perfoming a JOIN, it makes the query a little slower. I’ll have to look around to see whether there’s a more effective way of handling these queries.

Bmestep, I like your write up on using rsyslogd with PLA, do you mind if I put a link to your write up on the PLA website?

Kris

Look for a new post to capture ideas on enhancing speed / performance of PLA.

I’ve spent some time trying to speedup PLA without good results.
I think that the problem (in my case) is that we’re running “syslog server” and the database on the same system.
About the database changes, i’ve made changes to storage type on MySQL, adding partitioning and fixes (i hope) to some querys.
I changed expression:
log_time>’expr’ and log_time<’expr’
for
log_time between ‘time1′ and ‘time2′
while i’m using partitions by “range” (to_days) this sentences uses only partitions that contains records in this interval of time. The original sentences uses all partitions.

Now i’m working to move the database and the front-end to another system.

I hope that we’re having news about PLA!.