AI Penetration Testing: What You Should Know Before Choosing A Vendor

AI has made its way into security testing, much like other industries. It is now used in penetration testing, but the term does not refer to a standard service.

For instance, two vendors may both offer an AI penetration test while delivering substantially different assessments. The contrast lies in how much of the work AI performs and where human judgement enters the process. 

You will generally encounter three engagement models:

• A human tester leads the assessment and uses AI to support specific tasks.
• A human defines the scope and oversees the engagement, while AI performs most of the testing.
• AI conducts the assessment autonomously, without a person reviewing its findings.

Each model gives you a different level of human judgement, oversight, and validation. Unless the vendor explains its model clearly, the report you receive may not cover the risks you expected the test to address.

What Is A Penetration Test, And Why Does It Matter?

A penetration test is an authorized simulation of a cyberattack. You and the vendor agree which systems to test, such as your web application, API, mobile application, or network. You also document permission before testing begins.

The tester then looks for exploitable weaknesses and proves what they could allow an adversary to do.

Companies commission penetration tests for several reasons:

• A customer requires evidence of security before signing a contract.
• A compliance obligation, such as PCI DSS, requires penetration testing. During a SOC 2 audit, the report may also provide evidence that relevant security controls are working.
• Your company is preparing for a major product launch or infrastructure change.
• Your security team wants to understand its exposure before a threat actor or external researcher finds a weakness.

The immediate reason may vary, but the objective remains the same: identify exploitable weaknesses before they cause harm. The methods used against those systems are changing as well.

IBM’s 2025 Cost of a Data Breach research found that, on average, 16% of the breaches it studied involved attackers using AI. The same technology is now changing the test itself.

How Is AI Used In Penetration Testing? 

A human pentester and an AI penetration test bring different strengths to the same engagement:

Buyer consideration	Human-led penetration testing	AI-led penetration testing
Coverage within the testing window	A researcher selects and investigates the attack paths most likely to create meaningful security impact. The amount of testing is limited by the available specialist time.	AI can test multiple endpoints and vulnerability classes in parallel, increasing the breadth of the initial assessment.
Prioritization	The researcher can change priorities after examining the architecture, user workflows, and behaviour of the application.	AI follows the objectives, scope, and context provided to it. Incomplete inputs can affect which paths receive attention.
Product-specific behaviour	A researcher can examine role changes, approval flows, transaction sequences, and other workflows that may be individually valid but exploitable in combination.	AI is better suited to repeatable tests and established vulnerability classes than weaknesses that depend on understanding how the product is intended to operate.
Repeatability	Retesting preserves the researcher’s understanding of the application, but each additional round requires specialist time.	AI can repeat the same checks consistently after remediation or following a product release.

Most penetration tests move through four core steps:

• Map: Identify the systems in scope, including endpoints, user roles, authentication paths, and other entry points.
• Attack: Test those entry points for broken access controls, injection flaws, exposed credentials, and insecure configurations.
• Validate: Reproduce successful attempts and establish whether each weakness creates a genuine security impact.
• Report: Document the evidence, impact, affected systems, and recommended remediation.

Vendors may describe their approach as AI-driven, generative AI, or agentic testing. Agentic testing generally means that AI can plan and perform a sequence of testing tasks with limited human direction.

Speed has operational value because the window between vulnerability discovery and exploitation is shrinking. Mandiant’s M-Trends 2026 report estimates the mean time to exploit at minus seven days. In practical terms, exploitation can begin before a vendor releases a patch.

The speed and breadth offered by AI raise the next question: whether it can complete the full assessment without human oversight.

Can an AI Penetration Test Run Without Human Oversight? 

AI can already find and exploit real vulnerabilities. But can it finish a full penetration test without human supervision?

The figures below come from different benchmarks and time periods, but they all point in the same direction:

• AI can find flaws without being told where to look. A GPT-4 agent succeeded on 11 of 15 sandboxed web-vulnerability tasks when given up to five attempts per task. Open-source models scored 0%.
• Prior information changes the result sharply. GPT-4 exploited 87% of 15 vulnerabilities when given the public CVE description. Without it, the rate fell to 7%.
• Harder, multi-step work remains a clear limit. GPT-4o succeeded 20% of the time (3 of 15 tasks), while Llama 3.1-405B reached 26.7% (4 of 15). None of the models tested achieved unaided root access on a complete machine
• Human assistance materially improved the outcome. In AutoPenBench, an autonomous agent achieved a 21% success rate. Human assistance raised it to 64%.
• Real-world use shows why validation matters. Anthropic documented a cyber espionage campaign in which Claude performed approximately 80% to 90% of the tactical work. The system also “frequently overstated findings and occasionally fabricated data.”
  
  The campaign involved cyber espionage rather than an authorized penetration test, but the same validation risk applies: unverified findings cannot be treated as proven.

For a buyer, the requirement is straightforward: a named person should reproduce and approve every finding before it reaches the final report.

How Do You Choose An AI Penetration Testing Vendor?

Engagement Model	Testing Execution	Finding Validation	Best Fit For
Autonomous AI	100% AI-Driven	None / Unverified	Quick surface-level scans & triage
AI-Led, Human-Reviewed	Mostly AI-Driven	Sampling / Post-test review	High-volume basic checks
Human-Led, Machine-Assisted (CredShields)	AI-Accelerated	100% Human Reproduced	Audit compliance (SOC 2, PCI DSS) & critical apps

The phrase “human in the loop” is too broad to explain how an engagement actually works. A person may be involved at several different levels:

• Scoping only: A person defines the systems and boundaries. AI then performs the test and produces the findings.
• Final review: AI performs the testing. A person reviews its output after the work is complete.
• Continuous validation: A person works alongside AI throughout the engagement and reproduces each finding before it enters the final report.

Only continuous validation places a human reviewer between the AI’s output and every finding delivered to you.

Three questions help establish the level of oversight:

1. Who is the named person responsible for reviewing the findings?
2. Does that person reproduce every finding, or only review the completed report?
3. Does the report include the evidence and steps needed to verify how each finding was reached?

CredShields uses a human led, machine assisted model. A named security researcher reproduces each finding and approves it before it enters the final report. The report includes the steps and evidence needed to verify how the finding was reached.

Scope also needs to be confirmed before the engagement begins, particularly if the test will support an audit. 

A company preparing for SOC 2 should understand what to expect from penetration testing before the audit and confirm that the vendor’s scope will provide the necessary evidence.

Ask the vendor:

• Does the engagement cover web applications, APIs, and mobile applications?
• Does it test business logic and actual product behaviour alongside known vulnerability classes?
• Does the engagement include a retest after remediation?

A client, described its experience this way: 

“We engaged CredShields for a comprehensive penetration test covering our web application, REST API, and iOS and Android mobile apps. The entire engagement, from initial scoping to final certificate, was completed in under four weeks. For any startup or scaleup needing a credible, fast, and well-priced security assessment, CredShields is a strong recommendation.”

When the product being tested uses AI, the assessment has an additional layer. The surrounding application requires conventional security testing, while the AI components introduce risks of their own. That is the next question to examine.

What Should You Test When Your Product Uses Generative AI? 

If your product includes a chatbot, copilot, or AI agent, the assessment should cover both the surrounding application and the AI feature.

The AI-specific scope should include the instructions governing model behaviour, the data it can retrieve, how its output is handled, the tools or APIs it can use, and the permissions applied to each action.

Explicit testing is required when untrusted content can influence these components, restricted information is within reach, or the model can act on a user’s behalf.

How Should AI-Specific Attack Paths Be Tested?

The assessment should examine how attacker-controlled content moves through the AI feature and whether it can affect the data, systems, or actions connected to it.

OWASP Top 10 for Large Language Model Applications provides a useful structure for these tests:

• Prompt injection (LLM01:2025): An attacker may enter an instruction directly or hide it within a document, webpage, email, or other source processed by the model.
  
  Testing should determine whether the instruction can redirect the model and whether independent authorization controls still prevent access to restricted data or tools.
  
• Improper output handling (LLM05:2025): A model response can become dangerous when the application treats it as trusted content or code.
  
  Testing should establish whether generated output is validated before it reaches another component. Output rendered as active browser content may lead to cross-site scripting. Passing it into an executable backend function may result in remote code execution.
  
• Excessive agency (LLM06:2025): An AI agent may be able to follow links, call APIs, or perform actions through connected tools.
  
  Testing should establish whether it can be manipulated into sending unauthorized requests to internal services through server-side request forgery, calling privileged APIs, or performing actions beyond the user’s privileges. Each tool should enforce the user’s permissions independently of what the model requests.
  
• Vector and embedding weaknesses (LLM08:2025): A retrieval-augmented generation (RAG) system uses external information, often stored in a vector database, to provide context before the model answers.
  
  Testing should determine whether poisoned source content or manipulated embeddings used to retrieve it can change the information supplied to the model. It should also establish whether retrieval controls prevent data from crossing user, account, or tenant boundaries.

For a buyer, the practical question is which of these paths apply to the product and what evidence the report will provide for each one. 

The evidence should show how the model responded, which independent control failed or remained effective, and what data, system, or action was affected. A changed model response alone does not establish an exploitable weakness.

CredShields tests these AI-specific paths alongside conventional application surfaces so that the assessment covers model behaviour and the controls that determine its security impact.

What Should You Confirm Before Commissioning An AI Penetration Test? 

Before commissioning the test, establish what the report must enable you to do. It may need to satisfy a customer, support an audit, inform a product launch, or guide an internal security decision. That purpose should determine the final scope and deliverables.

The agreement should provide clear, written answers to four questions:

• What will be tested? The scope should identify the included systems, environments, and user roles, along with anything explicitly excluded. For an AI-enabled product, it should distinguish between conventional application surfaces and AI-specific components.
• Who is accountable for the findings? The engagement should name the person responsible for reviewing the assessment and approving the results delivered to you.
• What will you receive? The vendor should define the report contents, supporting evidence, remediation guidance, and any documentation required by the customer or auditor requesting the test.
• What happens after the report? Confirm how remediation questions will be handled, whether fixes will be retested, and what updated documentation will be issued after verification.

These commitments give you a consistent basis for comparing vendors. Speed, automation, and price become meaningful considerations once the engagement can demonstrate sufficient coverage, accountability, and usable evidence.

CredShields can walk you through a reproduced finding before you commit to an engagement. You can examine the evidence, understand the security impact, and see how the result was validated.

See what a validated finding looks like A CredShields researcher will walk you through a reproduced finding, the supporting evidence, and its security impact. Talk to a researcher

Frequently Asked Questions About AI Penetration Test

1. How often should you run an AI penetration test?

Testing frequency should follow risk, material changes, and any contractual or compliance obligations. Schedule a penetration test before a major launch and after significant changes to authentication, infrastructure, data flows, or externally accessible functionality.

For a product using generative AI, a material change may include connecting a new model, data source, API, or tool. Changes to the actions an AI agent can perform may also alter the test scope.

PCI DSS provides one concrete compliance benchmark. Requirements 11.4.2 and 11.4.3 require internal and external penetration testing at least once every 12 months and after any significant infrastructure or application upgrade or change. 

2. Which standards and frameworks apply to AI penetration testing?

Several standards and frameworks may inform the engagement, but they serve different purposes:

• The OWASP Top 10 for Large Language Model Applications provides a risk taxonomy that can be translated into test cases for generative AI applications.
• MITRE ATLAS catalogues the tactics and techniques adversaries use against AI-enabled systems.
• The NIST AI Risk Management Framework helps organizations govern, map, measure, and manage AI risks across the product lifecycle.
• ISO/IEC 42001 specifies requirements for establishing and maintaining an AI management system.

No single framework defines the complete penetration-testing scope. The vendor must translate the relevant guidance into specific systems, attack paths, permission boundaries, and evidence requirements for the product being assessed.

3. Are free or open-source AI penetration testing tools enough?

No, not for a complete penetration test. Free and open-source tools can support initial checks, targeted test cases, and regression testing. However, they cannot independently demonstrate full scope coverage, test product-specific business logic, or confirm the impact of every finding. They are best used as part of a validated assessment. 

Additional Read

The OWASP Smart Contract Top 10 (2026): A Production-Focused Security Implementation Guide