Vulnerability Assessments versus Penetration Testing
The bartender says to an empty chair “we don’t serve time travelers here.” Moments later, a time traveler walks into the bar and sits down in the chair.
Timing is everything both in bartending and security, so why do so many organizations do penetration testing before they have mastered vulnerability management? Two reasons. One, some security services providers conflate the vulnerability assessments with penetration testing, selling the reports from automated software scans as penetration testing. Two, penetration testing is more precise, so it is considered to be “better”, when it is not better but rather the refined part of a two-part process.
Vulnerability Management is the first, foundational step. Just to keep our terms straight, Vulnerability Management is the continuous process of using a software tool to do vulnerability assessments. The NIST guidance on how often an organization should scan is: “vulnerabilities to be scanned need to be readily updated as new vulnerabilities are discovered, announced, and scanning methods developed. This updating process helps to ensure that potential vulnerabilities in the information system are identified and addressed as quickly as possible.” Vulnerability management is the cyclical practice of identifying, classifying, prioritizing, remediating, and mitigating the effect of software vulnerabilities. Protecting against these vulnerabilities is done with a variety of responses such as installing patches, changing network security policies, reconfiguring software, or educating users about social engineering.
As SecOps author Dan Miesller writes, vulnerability assessments are the first step because “it is not for testing mature defenses, per se, but rather giving you a list of all the stuff you need to fix to have good defenses. It is a prioritized list of everything you should fix.” Think of the vulnerability assessment as a list of all the ways a criminal could get into the bank, and a pen tester demonstrates what the criminals would do once they are in.
Prioritization is key, and that is what good vulnerability management tools do well in the right hands. One estimate is that each year there are more than 16,000 identified vulnerabilities of which it is estimated that over 1,000 are exploited. Dynamically scanning your entire asset inventory and mapping against known vulnerabilities is obviously work for a machine. This generates a laundry list of vulnerabilities on risk, not just based on CVSS or vulnerability risk, but in context of threats, mitigation factors such as asset status, and configuration posture. This is where a specialist comes in, either a security team member or a managed vulnerability service, taking the laundry list and turning it into prioritized actions by quickly patching vulnerabilities or taking other remediation actions to reduce the attack surface. Today's security teams need to focus on speeding up the response, instead of needing to put in time and cost in managing integrations and solutions.
The goals of vulnerability management contrast with the goals of pen testing. The goal of a vulnerability assessment is to identify as many vulnerabilities as possible, whether important or unimportant. Breadth and volume. Pen testing is about precision, finding that combination of people, process and technology that is flawed, and that the automated tools missed.
Penetration tests have a critical role. Sometimes the combination of vulnerability assessments and penetration tests are mandatory. For example, in the recent past companies could comply with older versions of the standard by largely just doing an automated vulnerability assessment. In PCI DSS version 3.2, the requirements specify companies implement a penetration testing methodology and say companies must “validate segmentation,” which can only be done by performing a manual penetration test.
Seth Glasgow of IBM X-Force Security summed it up beautifully, “to cover all your bases, it’s best to use a combination of manual penetration testing and vulnerability assessments. I like to compare it to clubs in a golf bag. Not every club is needed for every shot, but to play the whole game, you need all of them.”
GlassHouse Systems provides a wide-range of security solutions and services including Vulnerability Management as a Service and Penetration testing.