Penetration testing is an exercise in trust. You’re handing a third party the means to access your systems, in many cases your source code, and the details of exactly where your product or company is vulnerable.
Before you sign off on an engagement, it’s worth understanding what that trust is actually built on, and what questions you should be asking.
Who is doing the work, and can you vet them?
Choosing a pen test provider isn’t just deciding which company logo you want on your testing report. Testing involves specific people accessing your systems and, often, your source code. You should know who those people are, what they’re qualified in, and how they’ve been vetted.
Are they contactable directly if a question comes up mid-engagement, do you know who is working on your testing, or is it just someone from an anonymous pool of testers? If a provider can’t or won’t tell you who’s doing the work, that’s an answer in itself.
How is the testing being done?
There’s a real difference between testing undertaken to a standard and more generic testing, often undertaken to a budget.
Standards-based testing, aligned to recognised methodologies and frameworks, means the engagement follows a defined, repeatable process rather than whatever a given tester feels like covering in the time available. Anything less than standards-based is, in effect, a compromise dressed up as a deliverable. It is technically a pen test, but it might not give you the assurance that you need.
Consider the role of AI
Leveraging AI capability is a given in almost any modern pen test engagement, but has the AI tooling been carefully locked down, or is there a risk that it’s been handed sensitive information about your systems or your source code? And is the AI being used as a tool to enhance human-led, creative exploits and lateral thinking – or a wholesale (potentially problematic) replacement for those skills?
Has the engagement been right-sized for you?
A good provider takes the time to understand your business and needs, and they scope to your actual risk and systems, not to what’s easiest to sell or quickest to deliver.
Over-selling looks like scope you don’t need and won’t use, and under-selling looks like a fixed-fee setup that quite possibly skips the harder, more time-consuming and manual parts of your attack surface.
Have you been sold vulnerability scanning under the guise of pen testing? The two are quite different and shouldn’t be confused.
Either way, you’re not getting what you think you’re paying for. The right scope should map to your assets and your risk profile, and a reputable provider will be willing to explain, in plain terms, why the scope looks the way it does.
How secure is the testing company itself?
This one gets overlooked more than it ideally should. During an engagement, a testing provider will accumulate some of the most sensitive information that exists about your organisation: live vulnerabilities, in your systems, often unpatched, for the duration of the test and beyond.
How and where is that sensitive information stored? Is it encrypted at rest and in transit? Who has access to it, and for how long is it retained after the report is delivered? A provider that can’t answer these questions with confidence is, ironically, a security risk in its own right.
Are the findings actually actionable?
A report full of CVSS scores and generic remediation advice isn’t much use to a team that has to prioritise real work against real deadlines. Findings should be specific to your systems, clear about actual impact, and broad enough to cover the attack surfaces that matter (not just the easy, well-trodden ones). If the recommendations could apply to almost anyone’s application, they’re not doing much for yours.
Will it hold up under scrutiny?
Ultimately, a pen test is often produced for an audience beyond your own security team, perhaps a board, an auditor, a customer’s due diligence process, or a regulator. The assurance it provides needs to be defensible: methodology-backed, professionally presented, and able to withstand someone else asking hard questions about how it was produced. If it can’t survive that scrutiny, it isn’t assurance. It’s paperwork, or a gimmick.
The bottom line
Trust in a pen testing relationship isn’t assumed, it’s earned through transparency: about who’s doing the work, how they’re doing it, how they’re protecting what they find, and whether what you get at the end actually holds up. If your current provider can’t answer these questions easily, it might be worth asking why.
