Solutions

SOC 2 Penetration Testing

Q: Autonomous vs. automated penetration testing: what's the difference?

The terms overlap, but 'automated' often means a scanner or scripted tool, while 'autonomous' means a system that reasons about the specific application, plans multi-step attacks, and validates by exploiting, closer to a human pentester than a scan. With Sybil, pentests can also be scheduled to kick off automatically at a set cadence.

SOC 2 doesn't name penetration testing as a requirement. Auditors expect one anyway. Here's what a SOC 2 pentest is expected to cover, and how to get an auditor-ready report without derailing your audit timeline.

Sybil runs SOC 2 pentesting for teams like Notion and Cursor. Cursor's engagement ran start to finish in two weeks.

RunSybil is backed by $40M from Khosla Ventures.You'll know exactly what evidence your auditor expects, and how to get it before your audit window closes.

Talk to us about scoping

See a sample finding

Trusted by security & engineering teams — including at Carta

The default engagement: Black Box

Black box: attacks like a real attacker would.

No credentials. No source code. No architecture diagrams. Sybil gets the same starting line a real adversary gets — the target, and the rules of engagement. Nothing else.

Account

Orders

Order History

3 orders • Showing your recent orders

#8128

Delivered

Wool Overcoat

Size L • Charcoal

Ordered Nov 12, 2024

$320.00

View Order

#8129

Delivered

Canvas Tote

Natural

Ordered Nov 3, 2024

$48.00

View Order

#8841

Not this account's order.

Leather Boot

Meridian Retail Group

Ordered Oct 28, 2024

$180.00

View Order

>_ Candidate-finding

Trying session tokens... Walking order IDs... Found accessible order: #8841 (Not this account's order)

Starts from:

Nothing but the target and the rules of engagement.

Proves:

What an anonymous or low-privilege attacker can actually reach and do from outside your perimeter.

Why it's the default:

It's the threat model most teams are actually buying a pentest to rule out — and what most compliance checklists mean by "penetration testing."

Need to test what's inside the code, not just what's reachable from outside it? White box is available too — more on that further down.

min

average breakout time in 2025: how long it takes a real attacker to go from first access to moving deeper into your network.

(CrowdStrike, 2026 Global Threat Report)

89%

year-over-year increase in attacks from autonomous adversaries. The other side of this industry already went agentic.

(CrowdStrike, 2026 Global Threat Report)

days

median time it actually takes to fix a pentest finding once it's reported. The bottleneck was never how fast the test runs — it's what happens after.

(BrightDefense, Penetration Testing Statistics, 2026)

It runs the engagement, not a checklist.

Every Sybil engagement moves through the same pipeline, whether it's a one-time pentest or ongoing coverage — access confirmed, surface mapped, techniques chained, findings independently reproduced, fixes shipped back.

01. Access & Recon

02. Plan & Execute

03. Identify

04. Validate & Escalate

05. Remediate & Retest

Access & recon

Credentials and scope are confirmed, then every reachable endpoint, page, and API inside that scope gets mapped.

Screenshot showing a web server response with HTTP 200 OK status and related headers in a black interface.

/admin/polls/question/?q=zap

Response

Request

Response headers

HTTP/1.1 200 OK Server: nginx/1.14.2 Date: Fri, 09 Oct 2020 21:43:04 GMT Content-Type: text/html; charset=utf-8 Content-Length: 5088 Connection: keep-alive Expires: Fri, 09 Oct 2020 21:43:04 GMT Cache-Control: max-age=0, no-cache, no-store, must-revalidate, private Vary: Cookie X-Frame-Options: DENY X-Content-Type-Options: nosniff Set-Cookie: csrftoken=K4tvsNT4IAs8recg8m1bLf

Plan & execute

Testing is prioritized by risk, then real attack techniques get run against the target — not a fixed checklist.

Attack Plan list showing categories like Admin Access Control & Broken Auth, RBAC & Multi Tenant Authorization, all marked Active.

Identify

Anything that looks like a real weakness gets flagged as a candidate finding.

Findings table listing vulnerabilities with severity, CVSS score, status, and test type.

Validate & escalate

A separate pass independently reproduces it into a canonical finding, then chains it to see how far a real attacker could actually get.

Security report showing SQL injection attack evidence on login endpoint with unusual database query patterns.

/api/v1/findings/4471/validate

Response

Request

Response headers

HTTP/1.1 200 OK Server: nginx/1.14.2 Date: Fri, 09 Oct 2020 21:52:03 GMT Content-Type: application/json Content-Length: 866 X-Finding-Status: confirmed X-False-Positive: false Cache-Control: no-store

Remediate & retest

Fix guidance ships precise enough to hand an autonomous coding assistant, with retesting available to confirm it holds.

Timeline showing three completed steps: Discovered, Finding Reported on 8/5/2026, and Closed (Fixed).

/api/v1/findings/4471/remediate

Response

Request

Response headers

HTTP/1.1 200 OK Server: nginx/1.14.2 Date: Fri, 09 Oct 2020 22:03:41 GMT Content-Type: application/json Content-Length: 512 X-Remediation-Status: patched X-Retest-Result: pass Connection: keep-alive

also available: White Box

White box: go deeper by exposing the agents to the source.

Black box proves what an outsider can reach. Some questions live deeper than the perimeter — broken authorization logic, internal APIs, role boundaries that only show up once you can read the code. That's white box.

Option 02

White Box: "The architect"

Starts from:

Full source access, credentials across roles, and infra/config — the blueprints, not just the walls.

Proves:

The deep, structural issues that only surface once you can read the code — broken authorization logic, internal API paths, role boundaries.

Best for:

Authorization logic, internal APIs, and anything where "did we build this right" matters more than "can someone find the front door."

Need something in between? We also run Gray Box engagements — standard, logged-in user access — to test how far a normal account, or one that's been stolen, could actually get. Run black box first, then white box to chase down what it couldn't see from outside — a white box pass often confirms exactly what a black box finding was pointing at. Either way, findings feed the same coverage picture, and either engagement can convert into ongoing, continuous coverage once you're ready to stop scoping this once a year.

Code example callout — white box engagement, flagged by Sybil

api/routes/invoices.py

def get_invoice(invoice_id): if current_user.id == invoice.owner_id: if current_user.org_id == invoice.org_id: return serialize(invoice) abort(403)

What we found:

The ownership check was widened from user-level to org-level in a recent change. Any member of an org can now reach every invoice in it — a new authorization boundary, not just a bigger blast radius.

How white box confirmed it:

Reading the code pointed straight at the check; testing every role and org combination against it confirmed the gap holds in practice, not just on the page.

What was real — canonical finding:

Confirmed: a low-privilege account in Org A enumerates invoice IDs and reads Org B's data through a shared billing proxy. — CWE-639 · CVSS 8.1 HIGH

The fix:

Remediation shipped precise enough to hand to an autonomous coding assistant: scoped query + regression test. Retest available to confirm it holds once merged.

What Sybil actually tests

Real applications, authenticated on day one.

Supported targets

Web apps (SPA / MPA) · REST APIs (OpenAPI ingestion) · GraphQL · gRPC · WebSockets · React / Angular / Vue · Mobile backends (iOS / Android API layer) · Internally exposed applications

Authenticated, day one

Username / password · Session cookies & custom headers · 2FA (TOTP, email codes, magic links) · SSO (SAML, OAuth2, OIDC) · RBAC across multiple roles · Multi-tenant applications

What this is built on

Four commitments, and one we think the industry should adopt.

Depth without the three-week wait

Same rigor as a manual engagement, without weeks lost to scoping calls and tester availability. Engagements start the week you sign.

Your access level, your call

Black box, white box, gray box, or a mix — without hiring two vendors or running two separate playbooks.

Test and prove, not just scan

Every finding ships with a working exploit and independent reproduction, not a severity score someone else has to argue with.

Gartner’s own definition of Adversarial Exposure Validation

The fix ships with the finding

Validated results return as remediation guidance precise enough to hand an autonomous coding assistant — not a PDF appendix nobody re-opens.

The one we think should be industry standard

One report, continuously true

Every engagement updates the same coverage picture instead of starting from zero next time, so the report your auditors, your customers, and your board see is still accurate when they read it — not just on the day it was written.

Also satisfies the requirement for

The report your auditor, customer, or insurer needs.

SOC 2 Type II

Pre-release / annual pentest requirement

Customer security questionnaires

Cyber insurance renewal

SOC 2 Type II

Sybil itself runs under SOC 2 Type II — independently audited over time, not assessed once.

Per-customer isolation

Scope, findings, and credentials are segmented per customer. Nothing crosses organization boundaries.

Hosted on AWS

Data encrypted both in flight and at rest.

Is this you

Built for teams who need a real test, on their timeline.

Need a pentest report for a customer, auditor, or insurer before a deal closes

Already outnumbered by the engineers actually writing the code

Shipping code faster than a boutique firm can schedule you in

Want an engagement that ends in a merged fix, not a 40-page PDF

Small security team, no bench to hand-test every new endpoint

Considering black box or white box and not sure which answers your actual question

Faq

Frequently Asked Questions

What is autonomous penetration testing?

Autonomous penetration testing is authorized offensive security testing performed by AI agents that continuously reason across live applications and infrastructure: discovering attack surface, planning and executing safe exploit attempts, adapting based on system behavior, validating real impact, and producing evidence-backed remediation guidance, without requiring a human in the loop.

Autonomous vs. automated penetration testing: what's the difference?

The terms overlap, but "automated" often means a scanner or scripted tool, while "autonomous" means a system that reasons about the specific application, plans multi-step attacks, and validates by exploiting, closer to a human pentester than a scan. With Sybil, pentests can also be scheduled to kick off automatically at a set cadence.

How does autonomous penetration testing work?

It authenticates to the application, maps the attack surface, attacks by chaining weaknesses into exploit paths, and validates exploitability before reporting.

Is autonomous penetration testing just a vulnerability scanner?

No. A scanner matches signatures and flags potential issues. Autonomous penetration testing reasons about your application and proves exploitability by chaining real attacks. Sybil reports only what it has confirmed is exploitable.

Can autonomous penetration testing replace manual penetration testing?

Yes. Sybil covers privilege escalation, multi-tenant isolation, and business-logic flaws that scanners miss, with capabilities continuing to expand for web applications.

Is autonomous penetration testing accurate, or does it produce false positives?

Accuracy depends on validation. Sybil runs every finding through a multi-agent pipeline that drives false positives toward zero.

Does autonomous penetration testing satisfy SOC 2 and ISO 27001 requirements?

Yes. RunSybil customers have satisfied SOC 2 Type II and ISO 27001 audits using Sybil's output, which ships pre-formatted with CVSS 3.1 scores and CWE IDs.

Do I need to share source code to run an autonomous penetration test?

No. Sybil is black-box by default; credentials and source code are both optional. Providing credentials enables deeper coverage, the same detail as a traditional third-party pentest. Source code is optional and enables deeper white-box coverage and code-level remediation.

Let's scope what black box or white box would find in your last release.

Bring us the app. We'll tell you honestly whether black box, white box, or both is the right way to test it — then run it in days, not quarters.

SEE HOW AN ENGAGEMENT RUNS

TALK TO US ABOUT SCOPING