AI's Evolving Competence in System Breaches

In a surprising encounter, co-founders Ariel Herbert-Voss and their team at a pioneering company were left momentarily taken aback when their advanced tool, Sybil, identified a significant flaw within a client's digital system last November.

Sybil employs a combination of various artificial intelligence techniques along with unique proprietary methods to meticulously examine IT structures for vulnerabilities exploitable by malicious actors, such as unaddressed security patches or incorrectly set-up databases.

On this occasion, Sybil highlighted a critical issue tied to the customer's use of federated GraphQL—an intricate language used in managing data requests online via APIs. This vulnerability inadvertently exposed sensitive information.

What intrigued Ionescu and Herbert-Voss was the sheer complexity involved in identifying the flaw, requiring comprehensive expertise across multiple system architectures and their interconnections. The team at RunSybil discovered similar issues in other GraphQL implementations, underscoring this as a logical leap in model capabilities—a leap they liken to a quantum shift.

This scenario highlights an increasing threat landscape. As AI technologies advance, their capacity to uncover zero-day vulnerabilities and other security risks grows substantially. The intelligence harnessed to identify these vulnerabilities is essentially the same intelligence that could be wielded to exploit them.

Advancements in AI, particularly those involving problem decomposition and agent-driven operations like web exploration and software manipulation, have significantly bolstered the cybersecurity acumen of latest AI models, according to a UC Berkeley expert specializing in AI and security.

“Recent developments signify a pivotal moment in cyber protection capabilities,” she asserted, illustrating the considerable enhancement seen over recent months.

In the previous year, a newly-developed benchmark known as CyberGym was introduced to evaluate the proficiency of large language models in identifying weak spots in major open-source software initiatives. This benchmark comprises 1,507 documented vulnerabilities spanning across 188 projects.

By July 2025, Anthropic’s Claude Sonnet 4 model had successfully identified around one-fifth of the benchmark's vulnerabilities. By October of the same year, an updated model, Claude Sonnet 4.5, increased this detection rate to thirty percent. As Song noted, AI agents are increasingly capable of pinpointing zero-day vulnerabilities efficiently and with minimal cost.

The trajectory of these advancements indicates a critical need for innovative safeguarding measures, which could involve enlisting AI as an ally to those defending digital infrastructures. “It's essential to explore different strategies for empowering AI to bolster cybersecurity defenses,” she stressed.

One suggested initiative involves major AI firms sharing their models pre-release with security professionals to enable proactive identification and remediation of potential issues.

Song also suggests reevaluating foundational software development practices. Experiments from her research group have demonstrated the potential for AI-generated code to exceed current programmer standards in security, advocating for a fundamental shift toward secure-by-design programming methodologies.

The team at RunSybil foresees that in the not-so-distant future, the advancement of AI's programming capabilities could tip the scales in favor of cyber attackers. Ariel Herbert-Voss reflects on the scenario, stating, “With AI's ability to produce executable commands and scripts, capabilities that hackers typically wield, this could accelerate offensive cyber tactics.”