6 Reasons You Are Wasting Money on Penetration Tests

Here's why you're not getting the value you deserve

On more occasions that I can count, I find myself trying to convince customers into rethinking their scope, ambition or even the type of penetration test they’ve ordered. While this usually works out fine, I’m not convinced that all pentest vendors go through the same detailed conversations.

In this article I’ll go outline six common reasons why the penetration test you paid for is lacking in value.

1. Your expectations are too low

Sadly, there are a lot of penetration test firms out there that really do glorified vulnerability assessments and call them penetration tests. This type of semi-scams probably happens in any profession, but with penetration tests being such a technical field, it is harder to detect for customers.

This is very understandable, but there are also good solutions to this. First of all, have a technical person with you during these discussions. And if you are the technical person, ask the pentesters about how they would test your environment. Sure, a pentester should not be on your level when it comes to configuring your infrastructure, but they should absolutely be able to tell you how it works, what vulnerabilities and misconfigurations often arise, and what type of methodology works well.

2. The test is performed by only junior testers

This is a big concern of mine. Firms showcasing senior talent during the sales pitch, only to have none of these people actually do the work on the project. This will lead to subpar testing and rough reports, despite what was promised.

Again, the solution is easy. If you talk to a tester that seems experienced, get it in writing in the contract that this person should perform a majority of the testing, and be the point of contact on the project.

This guarantees the expertise you’re paying for is actually being applied.

3. You are too specific on your requirements

Whenever a customer mentions “black box”, “red team” or “adversary emulation”, I know there is a high chance that they need to be saved from themselves.

This usually indicates that the customer might not fully understand what they’re asking for, which is of course understandable and okay.

If you don’t know much about pentesting, my advice would be to not try to force the test team into a specific set of rules that you think are the best. The Dunning Kruger Effect is unusually large for this. I especially see this with people that have higher degrees within information security and feel the need to show their authority when scoping the test.

Use language you are comfortable with to tell the test team what your concerns are, what you want to be tested, and what your ambitions are for the test.

Look for competent people and let them advise you on how the testing should be performed.

While a “black box” test might sound cool, it will probably just be a waste of money for you.

4. Not testing realistic attack scenarios

This is a big one. Most of the time, I have to pull out from the customer what attack scenarios they are afraid of, and what type of attack that would have the largest impact on their business.

One of the most crucial steps in penetration testing is identifying the real-world attack scenarios that fits your business, and then testing the likelihood of them happening, and what impact they would have.

While digging this out during the presale meeting works fine for me, not all pentest vendors ask these important questions. So you should be prepared to provide this information yourself. Have you hired third-party developers that are administrating some applications in your infrastructure? Have you outsourced the administration of your Azure tenant? Do you let employees connect to your company network through any device as long as they have the correct VPN credentials? Are you shaky on how to configure AD securely?

A penetration test is not a vulnerability assessment and will never be able to cover everything in your infrastructure unless you pay way too much money. It should provide general testing while also focusing on some attack scenarios that are the most likely to happen within your infrastructure.

5. The test type does not fit your maturity level

This is another thing that I see often. Buzzwords very often catch the attention of leaders in companies, and that could result in a lot of wasted money. The example that I see the most is “Red Teaming”.

Think of the maturity level for penetration testing as a pyramid, where you shouldn’t skip any of the tests. This is especially true when it comes to the vulnerability assessment. If you haven’t fixed the low-hanging fruits and are performing regular vulnerability scans, then the penetration test will be low-quality and often just expensive vulnerability assessments.

Whenever a customer asks me to perform penetration testing, and they don’t have the lower steps in place, I will advise them to delay perform the penetration tests, and instead give them a proposal for an often much cheaper vulnerability assessment.

6. Relying on Automated Penetration Testing

Good, automated penetration testing does not exist.

I know this might be controversial these days, and as a penetration tester myself I am of course biased. But this is very similar to how AI performs coding these days. The only developers afraid of losing their jobs to AI are bad developers.

A penetration test is tailored to likely attack scenarios in your infrastructure, performed by professionals that understand your business.

If any of these tools tried to actually perform penetration testing, they would quickly destroy your infrastructure. When I perform penetration test there is close communication before any exploitation, among other things to ensure that nothing important breaks.

In addition, these automated penetration testing tools will often give large outputs that needs specific knowledge to be able to prioritize and mitigate.

Many of the tools that call themselves “automated penetration testing” are cool and useful, but while they have their time and place, they are not close to replacing good penetration testers.

In conclusion… Use common sense

Most competent penetration testers aren’t just looking to earn a quick buck or find as many security flaws as possible. They want to help you improve your security posture.

Engaging with competent professionals who care about the outcome will not only increase the value of your test but will also ensure you’re getting the best result on your investment.

Focus on finding a test team that seem competent and have knowledge about your type of business and infrastructure, and let them guide you through how the test should be performed.

Of course, security people will always advise you to spend unlimited money on security. Be honest about your ambition levels, and the test team will find a type of testing that fits you well.

Stay Updated!

Thank you for reading! To stay updated and receive other posts I publish, subscribe to my mailing list: