I wanted to cover some WAF topics I haven’t seen covered much. Most WAF vendors talk about the security their product provides in terms of blocking attacks. I would like to delve into these WAF Blockings as well as mention some ideas for alternative uses for your WAF through it’s interactions with web clients.

Web Application Firewalls are interesting bits of technology. Depending on the product and deployment method you chose, they can transparently protect your web infrastructure using various protections by generating blocks when threats are identified. Depending on the product, they can Vulcan mind meld with your Apache instance, live as another F5 device in your network, take over a slotin your XBeam, or live life as a network appliance inside your datacenters.

This intelligent device COULD interact with the client in additional ways outside generating BLOCKs. For example: developers could leverage a WAF to provide additional protections, send notices to connect clients under specific conditions, or even prompt a client for confirmation before performing a specific function if certain criteria are met. After all the BLOCK a WAF generates doesn’t have to be a BLOCK at all, at least not in the context of traditional firewalls or even active-response IPS devices.

If WAF interaction with the client is a concern because you’re trying to keep your WAF invisible to the bad guys, you should know that that’s not a realistic expectation.

WAF’s block threats to your web applications identified through various security methods, but what does that mean?

There are a few options, largely dependent on the vendor and deployment method (transparent bridge, proxy, router, offline sniffing): TCP Reset, Request/Response DROP, out of band Reset via 3rd party. There’s no hard-fast requirement to only use a TCP Reset that’s sent to client and server, like IPS or active-response causing the TCP session/connection to be terminated, but this is controlled by deployment method.
The DROP method is like a virtual trapdoor inside the WAF where malicious traffic falls into a dark pit, never to be seen again.

Some WAF products can send a web coded response back to the web user inside their active session indicating their request could not be completed, some WAF can be configured to quarantine an IP Address or terminate a web session, in addition to dropping the client request or server response. The use of WAF generated error pages to interrupt and/or stop the web session alongside Request/Response dropping is more graceful than TCP Reset. Depending on your environment, TCP Resetting could create unexpected results on your web servers and typically this requires your WAF to be operating in Proxy mode.

In traditional transparent WAF deployments, these BLOCKs generated by a WAF are typically nothing more than a standard error page or a redirect to a logout sequence coded within the web application being protected.  Some WAF’s allow you to customize the page, insert scripting, and push it seamlessly to the end-user inside the existing SSL session. Alternatively, the client could be redirected to a destination within the protected application to log out their session, collect additional information, or open a support ticket (although the last one of those I saw, was more for looks than functionality).

If the WAF can generate web pages in response to client interactions inside an existing SSL session, the client would be interacting with the WAF. The Imperva Application Defence Center (ADC) has an interesting web fraud paper on enabling clients to interact with what I would describe as a security control panel, to help with CRSF/XRSF attacks and web fraud. I have played around with this a little and found some interesting uses – sorry saving that info for my next contracting gig!

The idea of using policies to trigger BLOCKs takes on a new meaning, if the WAF can be leveraged to a generate unique or controlled web pages when a specific policy is triggered or even redirect a user to a specific function inside an application if certain criteria are met, before continuing on inside an application. Don’t get me wrong, TCP Resets are good too – but this path offers much more robust options for a company from multiple perspectives.

Now the WAF can be used to not only BLOCK pure security-centric threats but also control the application behavior and client interaction if something fraudulent, abusive, or irregular is detected. For example you could leverage the behavior deviation capabilities of your WAF (profile violations) and construct a temporary input validation error handling process inside your WAF while your coders developed the handling inside the application. This would be a straight forward use of the acquired knowledge of the WAF, a simple error page containing the prohibited characters, and a method for the client to have a “do over” on the prior page.

Once again, the WAF is providing additional capabilities that an IDS/IPS cannot!

The traditional network security approach to securing your web servers and database servers is more than likely going to get you in trouble some day. Think about it. Network Security preaches deny everything and permit only what you need. Great, open up port 443 and send encrypted traffic to your web server. KaBOOM gotcha!

Think about your Web Application Firewall and the reasons for your investment in web application security.
Regardless of the technology you have selected, here are four protections your WAF investment needs to be providing:

#1 Enforce decryptable web communications.
This might seem counter-intuitive but first and foremost, if your WAF can’t see it – then the WAF can’t intelligently PROTECT your assets! You need to disable any encryption not supportedby your WAF. It’s a long-standing double-edge sword securing web communications but still being able to inspect the communications. No more pre-shared or temporary key SSL sessions, sorry Diffe-Hellman, most WAF’s only support pure RSA. In addition, this is a good time to make sure your servers negotiate at a respectable bit length.

#2 Enable Correlation.
Attack signatures are great, but correlation is better. If your WAF doesn’t offer some form of correlation of multiple signatures and security events before triggering an alert, you might consider picking one up that does. Web Intelligence is a good product, but it’s not an F5, Breach, or Imperva WAF, and that difference could cost you.

#3 Serve & Protect, becomes Learn & Protect.
The best offense is a good defense. If your WAF knows what the application it’s protecting looks like or even better, how it behaves, then the application’s very own structure, coding, and URL/parameter make-up becomes it’s shield against malicious attacks. You don’t need to wait for a signature to protect your web application from new SQL Injection or XSS or Fuzzing attacks, if the WAF is stopping anything that doesn’t conform to expected behavior!

#4 Assess THEN Customize.
When you build a new house, you might expect to have certain things done specific to your requirements before you ever set foot inside the house but you’ve at least looked at the blueprints and seen sketches of the final product. For a WAF guarding a Web Application, custom rules really should be the last thing you do, and ideally AFTER you validate existing protections aren’t enough through penetration testing or code scanning. The major WAF vendors support the inclusion of vulnerability assessments in their products for custom policy creation.

Obviously enabling any of these are subject to your risk exposure / tolerance, but I wouldn’t advocate running for any length of time without these protections regardless of the organization or the other protections you may in place to guard your web applications.

Consider what every online entity is up against, there is more money to be made hacking your protected assets by nefarious (hopefully external) sources than you have resources or funding – short of government entities. If that wasn’t bad enough there are newly coded applications and updates released every minute than there are security fixes going in. If you’re not fully leveraging what you have and not securing as you go, then your company is leaving something undone for the bad guys to come along and exploit.

How is your WAF being used? Is it being used? Need help getting more out of your WAF?

I had an email asking what placeholders I usefor logging platform integration. Rather than reply in a comment or email, I thought I’d just make a post out of the response.

Looking at placeholders, here are some of the ones I use the most:

• ${Alert.dn}  this is the alert id
• ${Alert.createTime} this is the time the ALERT was created (note this can be misleading)
• ${Alert.description} this is bound to the alert, so you may see “Distributed” or “Multiple” appended due to aggregation of events
• ${Event.dn} this is the event (violation) id
• ${Event.createTime} this is the time the EVENT was created (this is when the event happened}
• ${Event.struct.user.user} this is the username from a web or database action
• ${Event.sourceInfo.sourceIP}
• ${Event.sourceInfo.sourcePort}
• ${Event.sourceInfo.ipProtocol}
• ${Event.destInfo.serverIP}
• ${Event.destInfo.serverPort}
• ${Event.struct.networkDirection} which way is the traffic flowing that triggered the event?
• ${Rule.parent.displayName} this is the name of the Policy that was triggered

There are other placeholders you can leverage, but these are the core I start with. I like these because they’re used on the web gateway AND the database gateway. This lets me have a consistent intelligence feed to my log monitoring platform and my SIEM product.

The trick here is that I can see how may events roll up underneath a single Alert. In the syslog feed, I can track the duration of an attack as well as tell you when I last saw the activity, because I track Alert.createTime and Event.createTime.

There are lots of options for how you build your syslog feed:

• You may be interested in the response time of the query or web page
• Perhaps the response size is of concern to you
• You may treat threats differently depending on where they occur in a database table or URL
• You may be interested in the SOAP action or request

Last but not least, in addition to security events you can also push system level events in the same manner using different placeholders.

• Configuration events can be syslog’d on complete with the user making the change
• Gateway disconnect messages can be sent via syslog (snmp might be better, but you need to load the custom OIDs)
• Excessive CPU or traffic levels can be sent via syslog

How are you using placeholders?

I received some emails overnight on the Imperva DIY Syslog posting asking when to use the alert placeholders versus the event placeholders.

For anyone not familiar with the Imperva SecureSphere platform, the system has a handy feature that provides aggregation of events on the SecureSphere management server detected by the gateways. This works whether you’re using the web or database gateways but for today I want to focus on the relationship between the data coming from the gateways and the aggregated data on the manager,  I’ll let ImperViews get into the other details - you can read more in the Imperva documentation.

The first thing you have to take note of is the Imperva hierarchy for violations/events and alerts. When the Imperva detects a condition that meets the criteria of a policy, whether that’s correlation, signature, profile, custom, etc., a violation is triggered on the gateway and fed to the management server. Everything in the management server for reporting and monitoring builds off this violation/event detail from the gateway, the gateway is where the enforcement and detection takes place so that should make sense. This is how we know the gateway is taking action on our behalf!

Assuming you haven’t disabled aggregation on the SecureSphere settings, each violation is aggregated into an alert. There are several criteria that the management server uses when aggregating a violation, so you’ll want to check the documentation for your version. The basic idea is that the SecureSphere manager will aggregate similar violations against a server group, an IP Address, a URL, a policy, or some combination of thereof in a 12 hour window. An alert in SecureSphere will have at least one violation/event tied to it, but depending on your aggregation settings it may have more.

So???

So! When you push security events to an external log monitor, you have to decide if you just want the initial Alert information or if you want each violation that occurs! If you build the Action Interface using ALERT Placeholders you’ll only get the Alert data with no additional details in the underlying violation/event stream. This could be problematic, if you’re trying to figure out if something is still going on because remember the SecureSphere aggregates violations under a single Alert for up to 12 hours!

In addition to using the correct placeholders, you also have to enable the “Run on every event” checkbox in the Action Interface/Action Set.

I tend to mix the Alert and Event placeholders so that I get relevant Event details wrapped in the Alert context. I see no reason to make my logging solution work extra hard to establish the same correlation of the Events into Alerts that SecureSphere does automatically.

How do you manage your SecureSphere alerts and events?

I have had the fortune to support a few WAF installations, my preference is Imperva’s WAF solution. For any security product, being able to know what it’s doing and what is going on within the product is as important as the actual security being provided.

One of the features of Imperva’s solution that I find tremendously useful in an enterprise setting, and possibly an MSSP as well,  is the ability to construct custom syslog formats for triggered alerts and system events in almost any format. I like to think of this as a Do-It-Yourself syslog formatter because the feed can be built and sent anywhere, using any number of options. More importantly, the feed can be bundled with specific policies or event types to provide limitless notification possibilities that often require professional services engagements to develop and implement.

In Imperva terminology, any policy or event can be configured to trigger an “Action Set” containing specific format options for among other things syslog messaging. If your logging platform (PLA) or SIEM requires a specific format, there’s a very strong chance that, with no more effort than building a policy, you can build the ${AlertXXX} or ${EventXXX} constructs necessary for your needs.

You can model the alerts to look like the Cisco PIX format, ARCSight’s CEF format can be used, or you can make your own as I’ve done in this screenshot:

Basic Syslog Alert Format

In addition to allowing customized messaging format, Imperva’s SecureSphere platform allows unique message formats and destinations to be specified at the policy and event level. For example, a “Gateway Disconnect” or ” throughput of gateway IMPERVA-01 is 995 Mbps” message can be sent to the NOC’s syslog server for response, while XSS or SQL Injection policies can be directed to a SOC or MSSP for evaluation. Additionally, the “Action Set” policies can be setup so that the SOC is notified on both of the  messages above as well as security events.

The configuration of the custom logging format is very straightforward, using placeholders to build the desired message format.  The document ”Imperva Integration with ARCSight using Common Event Framework” provides a number of examples, including a walk-through for building a syslog alert for system events, standard firewall violations, as well as custom violations. The guide is directed at the integration with ARCSight.

Depending on the version of Imperva SecureSphereyou are running / evaluating, the alert aggregation behavior will differ. Newer versions (6.0.6+) better support SIEM platforms with updated alert details, where older versions push syslog events on the initial event only.

You can request a copy of Imperva Integration with ARCSight using Common Event Framework to get additional ideas on customizing your syslog feeds for your SIEM product.

I have had the fortune to support a few Imperva installations, alongside other WAF solutions. I would like to illustrate one use for logs available on the Impervaplatform that can be leveraged to augment website trend reports and monitor “exposure” on key URL’s.

If you’re not familiar with the Imperva platform, it is possible (as with other WAF vendor’s products) to build custom policies that must match specific criteria and upon triggering these events can feed data into various syslog feeds. The entire purpose of a WAF is to protect your web application from threats, although some argue this point, so it stands to reason there may be facets of a given web application that are more sensitive than others.

Take for example the check-out page for an online retailer where the customer enters credit card data and confirms their billing information. This location of a web application might benefit from heightened logging under certain conditions by a Web Application Firewall, such as: forced browsing, parameter tampering, XSS, Server Errors, etc. The application may be vulnerable to fraud activities, the business may want to keep a tab on who’s accessing these URLs, or there some other risk criteria than can be measured using this approach.

Traditional webserver logs will provide: client information such as user agent info, username, source ip, method, access URL, response time, response size, and response code. The logged data sits in the access log file on the specific web server by default, but this information is for the entire website.

The Imperva SecureSphere can provide some of the same information: username, IP, Port, user-agent info, accessed URL, response size, response time, etc – but in addition, the Imperva can track whether the session was authenticated, correlated database query (if you have Imperva database protection deployed), SOAP information, security details relevant to the specific policy. The kicker is that this can be sent in a format configured by the admin to a syslog listener in a format supported by web trend tools or SIEM products without engaging professional services.

I’m not advocating the replacement of web server logs for trend analysis, but I am suggesting the deployment of targeted logging for sensitive areas inside an application where this information would prove useful either in a fraud capacity, security monitoring capacity, or even in an end-to-end troubleshooting capacity where a WAF would have visibility beyond traditional network tools from the frontend of a N-tier web application. Deviations in response times, excessive response sizes, and unauthenticated access attempts to sensitive URLs are ideas that come to mind for leveraging the visibility a WAF can bring to the table.

I should disclose up front that I derive my living today supporting WAF technologies for a large corporation, and so it will come as no surprise that I have a few opinions on the use of WAF technology and in general how to go about protecting web applications.

Purists.
If you’re a purist and feel adamantly for or against Web Application Firewalls, I would urge you to consider the roots of defense-in-depth – just like the spoon in The Matrix, there is no silver bullet. OWASP’s concepts are as close as we’ll ever get to that silver bullet.

Secure Coding won’t get you out of every vulnerability and neither will a WAF, if for no other reason than the sheer complexity of the equipment needed to stand up web-enabled services introduces too many interdependencies to think every coder, developer, and vendor got everything right and there will never be a problem. — If you disagree with that, put down the Vendor Kool-Aid now before it’s too late.

Positive / Negative Security Models
Good grief.  Techie speak if ever there was any. Reminds me of the James Garner movie Tank, where little Billy is exposed to negative feedback in order to arrest his “bad” behavior. In my house, that’s called a spanking and you get one when it’s appropriate. My kids know what a spanking is and so does anyone reading this thread. Without googling, name two WAF products based on each of these Security Models: Positive & Negative — It’s okay, I’ll wait for you.

And we’re back…
On the topic of Security Models, I tend to think it takes a combination of protective technologies to provide any actual risk/threat  mitigation. I would personally like to see developers take advantage of a WAF’s ability to see how an application behaves. Moste developers don’t think of in terms of which web page does what, instead they’re working with APIs and objects. This is unfortunate because the rest of the world sees these applications as URL’s. The WAF can be that bridge to the developers. A WAF could in theory help the developer ensure that a specific sequence of events happens before a transaction is processed or prompt the client before transactions occur in specific instances to avoid CRSF.

To bring things back around to my original point. I do agree that the more complex a web application is and the more servers required to make a service available online, the more vulnerable and difficult to secure that application or service will be. I’m not sure who’s law that is but I’m sure one exists, complexity breeds more complexity.

No surprise there, if you are protecting a complex asset then it will be high maintenance – I said to put down the Kool-Aid, it’s for you own good – nothing is free!

A while back I started a series on Network Zoning and like most procrastinating, over-achievers: I got side-tracked (is that a self-induced form of ADD?) ! I have had the pleasure of interacting with a number of folks on the zoning topic, and so I wanted to take a moment to tack on an additional concept that doesn’t always get much attention but is very relevant in your network zoning design.

PERSPECTIVE and the impact of perspective.

Perspective in Network Zoning is a little like determine the perspective of an email without knowing the sender. If you’ve ever sent a witty email to someone who didn’t share your sense of humor, you’ve been impacted by perspective. Please be careful not to confuse perspective with context. Perspective deals with a vantage point, while a context is the surrounding details.

When zoning, the perspective of the actual components, users, and threats dictates a given device’s zoning requirements. Theoretically perspective actually defines the security posture.

Did that hurt? Just a little?

The configuration for each of these devices in this illustration is relative to their location in the network. Their perspective determines their configuration. Obvious right? Please keep in mind, the External Firewall or Internal Firewall could easily be a router with ACL’s

Consider that the External Firewall in this illustration sees untrusted incoming traffic and passes only traffic based on rules for the more-trusted networks.

This “trusted” traffic of the External Firewall is actually UNTRUSTED TRAFFIC for the Internal Firewall! After all this is the UNTRUSTED interface on the Internal Firewall.

The Internal firewall can be configured with the same blocking rules of the External Firewall in addition to new rules that are applicable to protecting the Internal Networks.

The addition or the difference in security configuration for internal or external firewalls will be controlled in-part due to perspective because you could obviously implement the same overall security policy on both firewalls but the expectation for what threats exist where will be based on perspective.

In the same light, your zones will have traffic or usage patterns and requirements relative to their placement in the network. External DNS servers will be configured and protected differently than Internal DNS servers. Network resources talking across zones will work differently than talking inside a zone. Your security practices and configuration will change accordingly. The configuration for a given zone will be driven by perspective – requirements will map out differently based on the perspective of users, threats, and policies.

Perspective will show up within the logs as well. When you review the logs on your devices, you will react differently to external threats to your internal servers logged on the actual internal server versus the External Firewall.

When you build out your network zone, be sure to keep perspective in mind. You may choose to overlap policies as a defense in depth practice, but please take care to define your zoning appropriately.

What’s your perspective?
Drop me a line and let me know!

PCI DSS called for implementation of code reviews and web-application firewalls (WAF’s) in order to continue compliance and fight off the Breach Boogieman. Organizations can also conduct code reviews, as outlined in section 6.

Some ‘experts’ believe the web firewalls are just another piece of technology being thrown on the bonfire, while others believe you will never find all the potential bugs and flaws in an organization’s custom code, let-alone commercial software.

Interestingly, there continue to be heated discussions debating the usefulness of WAF’s, where they have to be deployed, what they are supposed to inspect, and whether businesses should be distracted by WAF’s in the first place.  The most important aspect of all this is the functionality that is to be provided by this technology. The WAF requirements outlined in requirement 6.6:

• Verify that an application-layer firewall is in place in front of web-facing applications to detect and prevent web-based attacks.

Make sure any WAF implementation meets the full extent of the requirement because “detect and prevent web-based attacks” can get a little sticky. As technology goes, there are a few variations in how WAF’s have been developed. Some products use reverse proxying to interrupt the web session for the ‘detect’ and accomplish the ‘prevent’ by only allowing valid sessions. This validation is being done in variations just like typical IDS/IPS’s operation: you get your choice of signatures, anomaly detection, protocol inspection, and combinations thereof. Some of the available products skip the proxy function and monitor the web traffic like a traditional IDS/IPS for known or suspicious threats either in-line or via a SPAN or TAP. Companies can not only choose their type of technology but can also decide on using open-source software or commercially supported products or a cross between the two.

The open-source route offers mod_security for apache and if companies need commercial support, you can get an appliance running mod_security. I found it interesting, in a recent Oracle Application deployment, Oracle recommends the use of mod_security to service as an application-layer firewall and URL-filtering firewall for DMZ-deployments. If mod_security doesn’t fit your needs, Guardian is also an open-sourced software with detection and prevention capabilities. Both have commercial support and product options.

mod_security has some other interesting options. It is possible to take the SNORT web signatures and convert them to mod_security rules via a script provided with mod_security. There are also several groups that provide signatures / rules for mod_security to identity new threats.

Outside the open-source space, there are products like Imperva’s SecureSphere gateways that use anomaly detection and profiling to determine whether something should or should not be allowed to access a web server. This company’s product line features an interesting twist, the dynamic profiling technology relied upon to ‘detect and prevent’ comes from none other than the man that developed ’stateful packet inspection’ in CheckPoint firewalls.

Along with Imperva, are F5, Cisco, CheckPoint, and the usual list of security vendors ready to snatch up your “bail-out” funding . As with any security technology, only after a review of your organizations needs and a thorough pilot of the prospective technology will identify the best-fit for any organization.

At the end of the day, the use of WAF technology to mitigate web application security is but one of the many defenses an organization should have in place to provide data security and data privacy.

What do you use to guard the security of your web applications?

Recent questions, comments, and suggestions have prompted this post. I would like to collect ideas for improving PIX Logging Architecture or provide a place to point out issues with running PLA at your organization.

If you use PLA or another open-source tool, tell us how you solve logging latency, sluggish reporting, and other related bottleneck issues associated with centralized log collecting and monitoring.