Improving Application Performance with Faster Security Rules

As more application development teams integrate security into their workflows, they’re looking at the impact of security on their application — from an application pipeline, workflow and performance perspective.

Due to compliance requirements, web application and API protection (WAAP) is commonly deployed in an application stack. But as organizations move their workloads to the cloud and begin to embed security into their application’s fabric, the WAAP’s role is evolving. Organizations need WAAP platforms that can scale and adapt to the changing cloud architectures and growing security threats.

In this article, we’ll review how the WAAP engine — and, more specifically, a ruleset built specifically for use with that engine — can be adapted to scale into cloud workflows and improve the performance of your web application while maintaining strong protection from online threats.‍

Consider both the WAAP engine and its ruleset‍

The WAAP should be an important part of your application stack and network security program. Traditionally WAAPs were disconnected from application development and were viewed as a security layer that sat outside the application. But modern WAAP architectures and tools have helped SecOps collaborate with DevOps along the CI/CD process, including rule creation and testing as an application is developed. When used properly to analyze the traffic traversing your application infrastructure, the WAAP engine and its ruleset play an important role in how fast the analysis is completed and how well the resulting audit and block decisions perform as a part of the overall application architecture.

Like a car’s engine, computer, and drivetrain, a WAAP engine, its rulesets, and the application it protects should be designed and optimized to work together. A car manufacturer doesn’t design an engine in a bubble. It must be optimized for the rest of the car’s frame and components to reach peak performance and efficiency across the board. The same should hold true for the WAAP engine and its ruleset. A WAAP that doesn’t account for these variables can get overworked and overloaded and ultimately degrade the performance and user experience of the web application.‍

Reduce the WAAP workload to improve speed and effectiveness‍

The Edgio WAAP uses a combination of open-source, industry, and proprietary signatures to inform its ruleset. We carefully evaluate each signature’s impact on the performance of the overall ruleset.

As we designed and optimized our WAAP engine, we saw that by collecting and consuming the open-source and industry rulesets without any modifications, (nearly) every rule would need to be analyzed as the traffic crosses the wire. This model is fairly common amongst WAAPs and makes it “easy” to see what happens to the traffic. But it also forces the WAAP to run through every rule to evaluate the risk associated with the traffic. This puts an unnecessarily heavy workload on the WAAP that is further exacerbated as the ruleset grows.

Of course, it’s not always about speed. A key element in a WAAP is its ability to determine the risk (compromise) versus reward (successful transactions) associated with the traffic and decide whether to flag the traffic as malicious, suspicious, or benign. The WAAP would either send an alert to the team (audit/alert mode) and block or re-route the traffic if it is deemed too risky to let through (production/block mode).

This analysis becomes a bigger challenge when examining the ingested open source and industry rulesets. It’s unlikely they’ve been created with speed and efficiency in mind for every possible WAAP engine in the market. This doesn’t mean they’re bad, it just means they focus on risk first, leaving performance as an afterthought. In other words, there’s room for improvement.

With rulesets created for completeness and accuracy first, many WAAPs find they can’t handle the traffic being analyzed in both modes (audit and block), forcing InfoSec teams to pick a mode: start with audit mode and move it to production mode or going straight into production mode with little to no real-world testing, which leaves them open to missed threats and false positives.‍

How Edgio enhances its WAAP rulesets‍

The open-source OWASP ModSecurity model for rulesets provides a standard, consistent manner through which WAAP rulesets can be developed and implemented. To complement this, OWASP offers an industry generic ruleset that can be ingested by a WAAP that follows this model.

However, as our team was implementing the ruleset, they realized that the rulesets being ingested — while complete and accurate — were indeed holding back the WAAP engine. So, the team decided to examine how rulesets are structured and processed. What we found, which should not surprise any WAAP admin, is that many rules go through the same data over and over to cycle through the checks necessary to determine the riskiness of the traffic passing through the WAAP.

Some protocol enforcement rules are fairly straightforward and don’t have to “live” inside the ruleset ecosystem. To address this, we first developed the ruleset that effectively trimmed down the OWASP ruleset by implementing them directly into the WAAP engine in a non-ModSecurity format. This results in early execution and faster detection.

Next, we targeted rules that analyze request bodies. We built a caching layer that mitigates the performance overhead of recalculating the same expressions in the rules. Taking an example of XML format bodies, we searched how many times the expression “XML:/*” appears in the OWASP CRS ruleset to see how many times an XPath could be recomputed in the processing of a single request without a caching layer, the number is significant.

Finally, where a lot of macro expansions/string substitutions and explicit searches take place, we identified and removed those rules from the ruleset and implemented them in the engine with better lookup and search algorithms such as HashMap lookups and phrase match (pm) operator instead of a regex (Rx) operator wherever possible.

By doing things differently with the structure of the rulesets, our WAAP engine uses faster data structures and open standard formats like JSON, which are easier to configure and run faster. This results in better performance, not only in terms of speed but also in terms of accuracy:

• Low false positives: Edgio arranges and controls the ruleset in a way that ensures accuracy is maintained
• Enhanced engine performance: The Edgio WAAP engine runs the rulesets in a manner that avoids overtaxing the engine

For example, for the OWASP IP blocklist policy rule, the ModSecurity engine uses its own “msc_tree” algorithm that looks at a bad IP list. Alternatively, the Edgio WAAP engine uses a different algorithm that is quite a bit faster. The following test results show how Edgio WAAP performs against ModSecurity. ModSecurity adds about 200ms to 500ms latency when performing a million queries on big IP lists. Edgio WAAP handles around 10B requests per day across its network. These small numbers matter a lot on such a large scale.