Penetration testing puts a human adversary against a running system. It returns a list of confirmed, exploited vulnerabilities ranked by impact — proof rather than prediction. Where threat modeling asks what could go wrong at design time and fuzzing generates inputs at scale, a pentester reports what worked: a shell on the box, records pulled from a database, one identity escalated into another.

What it catches

• Exploitable chains, not isolated findings. The bug a scanner rates medium that yields remote code execution when chained with a second. The report is the chain that actually fired.
• Business-logic flaws. Abuse of legitimate features that no signature or type catches because each step is individually valid: price manipulation, workflow bypass, replay of a valid request.
• Authentication and authorization bypass. Session fixation, privilege escalation, a missing access check on an endpoint the design assumed was internal.
• Misconfiguration and exposure. A default credential, an exposed admin panel, an over-permissive cloud role — present in the deployed environment, invisible in source.

What penetration testing does not catch: anything outside the time-boxed scope or the tester's skill. Coverage is neither systematic nor repeatable — two testers find different things on the same target — so a clean report is evidence of effort, not of security. It complements, never replaces, the systematic methods that run on every change.

Automated, manual, triage

A real engagement composes all three activities rather than choosing one:

• Automated scanning (DAST, dynamic application security testing). Tools generate payloads against the running target: SQL injection, cross-site scripting, known-CVE probes. This is fuzzing applied to security — fast, broad, noisy.
• Manual exploitation. A human crafts and adapts the exploits a scanner cannot: chaining bugs, abusing business logic, defeating custom authentication. The oracle is self-evident — the breach is its own proof. The input has no fixed shape; it is whatever the target's responses suggest next, so the method is bounded by the tester, not by a corpus.
• Exploitability triage. Human judgment turns raw findings into a ranked report and filters scanner false positives: is this real, reachable, and worth fixing?

Tools and methodology

• Standards: NIST SP 800-115 (Scarfone et al. 2008)¹ for test methodology, the OWASP Web Security Testing Guide for web-app test cases, the Penetration Testing Execution Standard for engagement phases, and MITRE ATT&CK as a checklist of real adversary techniques.
• Scanning and proxies: OWASP ZAP and Burp Suite (intercepting web proxies), nuclei (template-driven vulnerability scanning), sqlmap (SQL-injection exploitation).
• Exploitation: Metasploit (exploit framework and payloads), plus the bespoke scripting each target demands.

When to use, when not

Use:

• Before launching an internet-facing system that handles money or sensitive data, and after a significant architecture or authentication change re-opens the attack surface.
• Where an assurance regime calls for an independent test: PCI DSS requires one, and SOC 2 audits and procurement gates commonly request a report.
• Once the cheap, systematic methods already run (types, static analysis, fuzzing) and the residual risk is the exploit chains and business logic a human finds.
• Starting from a current threat model: the engagement is faster and deeper when it begins from the design's own list of what matters.

Don't:

• As the first or only security method. A pentest that re-discovers what a linter or static analyzer would have caught is expensive duplication; run the cheap, repeatable methods first.
• As a one-time checkbox. Each test is a point-in-time snapshot of one scope; the next release re-opens it.
• Without budget to fix and re-test. An exploited finding left unremediated — or a fix that is never re-tested — leaves a documented breach path open; the report then records the gap instead of closing it.

Evidence

• Arkin, Stender, and McGraw (Arkin et al. 2005)² report that a penetration test identifies a small representative sample of a system's security risks rather than a complete list, and that passing one gives very little assurance that the application is immune to attack. A test's success depends on the tester's skill, knowledge, and experience, and on other factors that resist metrics and standardization. Their positive case is conditional: the tests must be driven by the risks tracked from the start of the lifecycle, and the findings fed back into development.
• The support is definitional rather than statistical: a successful exploit is hard proof that a vulnerability is real, while an unsuccessful engagement proves little.

Classification

• Quality dimensions: Security.
• Area: Internet-facing and security-sensitive systems, before launch or after a major change — web apps, APIs, network services, cloud deployments; red-team and bug-bounty engagements.
• Guarantee: Heuristic.

Referenced by

• Agent security testing · Methods
• Threat modeling · Methods

References

---

1. Scarfone, Karen, Murugiah Souppaya, Amanda Cody, and Angela Orebaugh. 2008. Technical Guide to Information Security Testing and Assessment. NIST Special Publication Nos. 800-115. National Institute of Standards; Technology. https://doi.org/10.6028/NIST.SP.800-115. ↩

2. Arkin, Brad, Scott Stender, and Gary McGraw. 2005. "Software Penetration Testing." IEEE Security & Privacy 3 (1): 84–87. https://doi.org/10.1109/MSP.2005.23. ↩