The difference between penetration testing and vulnerability scanning is a widely discussed topic, however lately I have noticed that there are still numerous grey areas for the end users, and namely customers. One particular reason for this could be that technical language used in describing those type of assessments, may be clear to the security specialists, but not to the customers themselves. Some times, even IT engineers with an overall knowledge on networks could get confused by raw definitions.

That being said, I figured out that the best way to describe it to the end user is by describing a real live scenario. In the next paragraphs I’ll try to take the customer’s perspective and answer the questions that every end user has in mind when trying to understand the difference between penetration testing and vulnerability scanning.

“What benefits do I get when choosing a penetration test over a vulnerability scan?”

I have a great example that will serve as a scenario for this question from one of our previous assessments.
However, let us start from the beginning. It is widely known that vulnerability scans are performed by automated tools and pentests are performed manually, so the most common misconception here is that those tools won’t be able find every vulnerability and gap that a penetration tester could. Well, won’t they? Vulnerability scans are actually a key element of a pentest and they aren’t just checking patch versions (anymore) – vulnerability scanners have become a very specialised tool because most of the tasks performed by pentesters can simply be automated. They will check default credentials, do brute forcing, fuzz inputs to discover any possible errors, and lots more. What a penetration tester then does is analysing the findings and eventually confirming them by exploiting the vulnerabilities.
The exploitation may take a pentester further and allow him to escalate the attacks to another machine that wasn’t directly possible to compromise from the network.

Why doesn’t the vulnerability scanner find it all?

As I mentioned above, I will show it through an example. During one of our assessments we found a system with the ability to have an anonymous FTP login. This could have been done by mistake or by intention. The folder was empty, there was no write permission and there was no possibility to go “up” in the directory structure. This would be discovered by most vulnerability scanners and, for example, Nessus Scanner reports this with medium severity (CVSS: 5.0). The remediation guide is to disable this if it was done by mistake, but if it was done on purpose the client will simply leave it as is. What was not discovered by the vulnerability scan is that this FTP server was vulnerable to directory traversal (the attacker could read files outside of home directory).

In summary, the vulnerability could allow the reading of the file providing a prope path. If the FTP user home directory is in C:\Users\anonymous attacker can read C:\boot.ini file by providing the path:

../../boot.ini

Now here is the key difference between vulnerability scanning and manual penetration testing. The scanners are able to find such vulnerabilities, but they will check only those default locations – the scanner will assume the “C:\Users\anonymous” is the home directory and will try to read boot.ini file in the way I explained above. If, however, the home directory is elsewhere, the scanner will not read the file and this will not discover the vulnerability. Now don’t be fooled that scanners are that limitless. They’ll check many different combinations of paths in order to discover this vulnerability and in most situations they will succeed. But then again a Nessus Scanner for example will report this as a medium vulnerability with 5.0 CVSS score.

We got to the heart of the matter!

So we have two medium vulnerabilities reported by the scanner. Anonymous users can login and potentially read some files but they don’t know what to look for, and trying to enumerate a whole directory structure can trigger some IDS system. This is a problem, but as reported by the scanner it is of medium or even low severity so as an end user I would not be too worried about this, especially because this is a developer computer so even if hackers read some files from it, there are no sensitive information – I might fix it when i have time.
What the scanner won’t tell the customer, is that those two “medium” vulnerabilities can be combined – a server path exposure in a web application error message gives the penetration tester the knowledge of what and where to look for.
This particular system had also had a password protected web application and the password for it was stored in a file. By utilising those previously discovered vulnerabilities I was able to download the password file through FTP anonymous access. After accessing this application it revealed that there is a script allowing me to upload files to this computer – and this means game over for every system. The computer got compromised allowing to escalate attacks to other, critical systems. Bang!

Conclusion

If you were not sure about the difference between vulnerability scanning and penetration testing before, you will definitely see the difference through the above example – it’s a couple of medium/low findings versus a critical vulnerability. But if the customer had fixed at least one of those minor issues the attacks would have not been possible.
It’s all about the approach to the information you receive. One of my colleagues have a policy not to use “Low severity” in their reports – all findings are at least medium – that’s one good way.
Another important conclusion from this example is that much of the difference between a penetration test and a vulnerability scan depends on customisation. This is how custom systems (mostly websites / web application) pose the greatest risks. You can simple run automated scans on those but be sure to treat all those medium/low severity findings seriously. If you are not sure how, find a skilled penetration tester.