Part I: Preparing the Battle Space
The art of war teaches us to rely not on the likelihood of the enemy’s not coming, but on our own readiness to receive him; not on the chance of his not attacking, but rather on the fact that we have made our position unassailable.
—Sun Tzu in The Art of War
“Is our web site secure?” If your company’s chief executive officer asked you this question, what would you say? If you respond in the affirmative, the CEO might say, “Prove it.” How do you provide tangible proof that your web applications are adequately protected? This section lists some sample responses and highlights the deficiencies of each. Here’s the first one:
Our web applications are secure because we are compliant with the Payment Card Industry Data Security Standard (PCI DSS).
PCI DSS, like most other regulations, is a minimum standard of due care. This means that achieving compliance does not make your site unhackable. PCI DSS is really all about risk transference (from the credit card companies to the merchants) rather than risk mitigation. If organizations do not truly embrace the concept of reducing risk by securing their environments above and beyond what PCI DSS specifies, the compliance process becomes nothing more than a checkbox paperwork exercise. Although PCI has some admirable aspects, keep this mantra in mind:
It is much easier to pass a PCI audit if you are secure than to be secure because you pass a PCI audit.
In a more general sense, regulations tend to suffer from the control-compliant philosophy. They are input-centric and do not actually analyze or monitor their effectiveness in operations. Richard Bejtlich, a respected security thought leader, brilliantly presented this interesting analogy on this topic:
Imagine a football (American-style) team that wants to measure their success during a particular season. Team management decides to measure the height and weight of each player. They time how fast the player runs the 40 yard dash. They note the college from which each player graduated. They collect many other statistics as well, then spend time debating which ones best indicate how successful the football team is. Should the center weigh over 300 pounds? Should the wide receivers have a shoe size of 11 or greater? Should players from the northwest be on the starting lineup? All of this seems perfectly rational to this team. An outsider looks at the situation and says: “Check the scoreboard! You’re down 42–7 and you have a 1–6 record. You guys are losers!”
This is the essence of input-centric versus output-aware security. Regardless of all the preparations, it is on the live production network where all your web security preparations will either pay off or crash and burn. Because development and staging areas rarely adequately mimic production environments, you do not truly know how your web application security will fare until it is accessible by untrusted clients.
Our web applications are secure because we have deployed commercial web security product(s).
This response is an unfortunate result of transitive belief in security. Just because a security vendor’s web site or product collateral says that the product will make your web application more secure does not in fact make it so. Security products, just like the applications they are protecting, have flaws if used incorrectly. There are also potential issues with mistakes in configuration and deployment, which may allow attackers to manipulate or evade detection.
Our web applications are secure because we use SSL.
Many e-commerce web sites prominently display an image seal. This indicates that the web site is secure because it uses a Secure Socket Layer (SSL) certificate purchased from a reputable certificate authority (CA). Use of an SSL signed certificate helps prevent the following attacks:
- Network sniffing. Without SSL, your data is sent across the network using an unencrypted channel. This means that anyone along the path can potentially sniff the traffic off the wire in clear text.
- Web site spoofing. Without a valid SSL site certificate, it is more difficult for attackers to attempt to use phishing sites that mimic the legitimate site.
The use of SSL does help mitigate these two issues, but it has one glaring weakness: The use of SSL does absolutely nothing to prevent a malicious user from directly attacking the web application itself. As a matter of fact, many attackers prefer to target SSL-enabled web applications because using this encrypted channel may hide their activities from other network-monitoring devices.
Our web applications are secure because we have alerts demonstrating that we blocked web attacks.
Evidence of blocked attack attempts is good but is not enough. When management asks if the web site is secure, it is really asking what the score of the game is. The CEO wants to know whether you are winning or losing the game of defending your web applications from compromise. In this sense, your response doesn’t answer the question. Again referencing Richard Bejtlich’s American football analogy, this is like someone asking you who won the Super Bowl, and you respond by citing game statistics such as number of plays, time of possession, and yards gained without telling him or her the final score! Not really answering the question is it? Although providing evidence of blocked attacks is a useful metric, management really wants to know if any successful attacks occurred.
With this concept as a backdrop, here are the web security metrics that I feel are most important for the production network and gauging how the web application’s security mechanisms are performing:
- Web transactions per day should be represented as a number (#). It establishes a baseline of web traffic and provides some perspective for the other metrics.
- Attacks detected (true positive) should be represented as both a number (#) and a percentage (%) of the total web transactions per day. This data is a general indicator of both malicious web traffic and security detection accuracy.
- Missed attacks (false negative) should be represented as both a number (#)and a percentage (%) of the total web transactions per day. This data is a general indicator of the effectiveness of security detection accuracy. This is the key metric that is missing when you attempt to provide the final score of the game.
- Blocked traffic (false positive) should be represented as both a number (#) and a percentage (%) of the total web transactions per day. This data is also a general indicator of the effectiveness of security detection accuracy. This is very important data for many organizations because blocking legitimate customer traffic may mean missed revenue. Organizations should have a method of accurately tracking false positive alerts that took disruptive actions on web transactions.
- Attack detection failure rate should be represented as a percentage (%). It is derived by adding false negatives and false positives and then dividing by true positives. This percentage gives the overall effectiveness of your web application security detection accuracy.
The attack detection failure rate provides data to better figure out the score of the game. Unfortunately, most organizations do not gather enough information to conduct this type of security metric analysis.
Our web applications are secure because we have not identified any abnormal behavior.
From a compromise perspective, identifying abnormal application behavior seems appropriate. The main deficiency with this response has to do with the data used to identify anomalies. Most organizations have failed to properly instrument their web applications to produce sufficient logging detail. Most web sites default to using the web server’s logging mechanisms, such as the Common Log Format (CLF). Here are two sample CLF log entries taken from the Apache web server:
109.70.36.102 - - [15/Feb/2012:09:08:16 -0500] "POST /wordpress//xmlrpc.php HTTP/1.1" 500 163 "-" "Wordpress Hash Grabber v2.0libwww-perl/6.02" 109.70.36.102 - - [15/Feb/2012:09:08:17 -0500] "POST /wordpress//xmlrpc.php HTTP/1.1" 200 613 "-" "Wordpress Hash Grabber v2.0libwww-perl/6.02"
Looking at this data, we can see a few indications of potential suspicious or abnormal behavior. The first is that the User-Agent field data shows a value for a known WordPress exploit program, WordPress Hash Grabber. The second indication is the returned HTTP status code tokens. The first entry results in a 500 Internal Server Error status code, and the second entry results in a 200 OK status code. What data in the first entry caused the web application to generate an error condition? We don’t know what parameter data was sent to the application because POST requests pass data in the request body rather than in a QUERY_STRING value that is logged by web servers in the CLF log. What data was returned within the response bodies of these transactions? These are important questions to answer, but CLF logs include only a small subset of the full transactional data. They do not, for instance, include other request headers such as cookies, POST re...