Unlocking Harm: How Sentence Pairing Exposes Language Model Weaknesses

Author: Denis Avetisyan

New research reveals a surprisingly effective method for prompting large language models to generate malicious code, even those considered highly secure.

SPELL streamlines large language model serving through speculative decoding, pre-drafting potential continuations <span class="katex-eq" data-katex-display="false"> \hat{y}_{t} </span> with a small language model before verifying them with a larger, more accurate one, thereby reducing latency and improving throughput even under high load. — SPELL streamlines large language model serving through speculative decoding, pre-drafting potential continuations $\hat{y}_{t}$ with a small language model before verifying them with a larger, more accurate one, thereby reducing latency and improving throughput even under high load.

SPELL, an automated framework leveraging sentence pairing, demonstrates significant vulnerabilities in state-of-the-art language models and proposes a simple intent-extraction defense.

While large language models empower developers and democratize software creation, this accessibility simultaneously presents a critical security risk by enabling the generation of malicious code. This paper introduces SPELL-*’Casting a SPELL: Sentence Pairing Exploration for LLM Limitation-breaking’-a novel automated framework demonstrating substantial success in jailbreaking state-of-the-art code models to produce harmful outputs. SPELL achieves high attack success rates across multiple models-exceeding 73% validation by established detection systems-by strategically combining sentence pairings to craft effective adversarial prompts. Given these demonstrated vulnerabilities, how can we effectively align AI safety mechanisms within code generation applications to mitigate the potential for malicious exploitation?

The Illusion of Safety: LLM Jailbreaks and the Persistent Threat

The rapid advancement of Large Language Models (LLMs) has unlocked unprecedented capabilities in natural language processing, yet this power is counterbalanced by a significant vulnerability: ‘jailbreak’ attacks. These attacks don’t involve hacking in the traditional sense, but rather the skillful manipulation of LLMs through specifically designed inputs known as ‘Adversarial Prompts’. These prompts, often subtly worded or cleverly disguised, circumvent the built-in safety protocols and ethical guidelines intended to prevent the model from generating harmful or inappropriate content. Essentially, these models, trained to be helpful and harmless, can be ‘tricked’ into producing outputs they were never intended to – ranging from biased statements and misinformation to instructions for illegal activities – highlighting a critical need for robust defense mechanisms against such adversarial manipulation.

The core vulnerability of large language models lies in their susceptibility to generating malicious code despite built-in safety protocols. Adversarial prompts, cleverly designed inputs, can effectively bypass these safeguards, compelling the model to produce outputs ranging from harmful instructions to functional malware. This isn’t a simple case of the model ‘refusing’ to answer; rather, the prompts subtly reframe the request, exploiting loopholes in the model’s training to achieve a desired, yet dangerous, outcome. Consequently, a seemingly innocuous interaction can be manipulated to generate code capable of phishing, data exfiltration, or even system compromise, highlighting a critical security risk as these models become increasingly integrated into sensitive applications and automated systems.

Current techniques for discovering vulnerabilities in Large Language Models through adversarial prompt generation present significant challenges in terms of computational resources. Methods such as genetic algorithms, while capable of evolving prompts to bypass safety protocols, require extensive iterations and processing power, making them slow and costly to implement. Similarly, training deep learning agents to craft these prompts demands substantial datasets and complex neural network architectures, further increasing the computational burden. This inefficiency hinders rapid vulnerability assessment and proactive defense development, as researchers struggle to keep pace with the evolving capabilities of LLMs and the creativity of potential attackers. Consequently, the search for more streamlined and cost-effective methods for generating adversarial prompts remains a critical area of research in the field of AI safety.

SPELL: Automating the Art of Deception

SPELL is an automated framework designed to generate malicious code through the dynamic assembly of sentence components. Unlike static prompt creation methods, SPELL constructs prompts by selecting and combining pre-defined elements from a knowledge base, allowing for a potentially limitless number of unique attack sequences. This approach contrasts with methods requiring extensive pre-training on specific attack patterns; SPELL aims to adapt to novel scenarios by intelligently reconfiguring existing components. The framework’s core functionality revolves around the automated selection and concatenation of these components to form complete, executable prompts intended to elicit malicious behavior from target language models.

SPELL’s adaptability to varied attack scenarios is achieved through the use of a ‘Prior Knowledge Dataset’ containing pre-defined sentence components and a ‘Time-Division Sentence Selection’ method. This technique divides the prompt generation process into discrete time slots, allowing the framework to strategically select and combine these components based on the evolving context of the attack. By dynamically assembling prompts from this dataset, SPELL circumvents the need for large-scale pre-training on specific attack types; instead, it leverages existing knowledge to rapidly construct effective prompts for diverse situations, reducing computational cost and increasing flexibility.

SPELL employs large language models (LLMs) – specifically GPT-4.1, Qwen2.5-Coder, and Claude-3.5 – as the core engine for generating malicious prompts. These LLMs are utilized to assemble attack instructions by combining pre-defined sentence components, effectively leveraging their natural language generation capabilities. The selection of these particular LLMs is based on their demonstrated proficiency in code generation and complex reasoning, which are crucial for constructing effective and adaptable attack prompts without requiring task-specific fine-tuning. The framework’s reliance on these models allows for the creation of diverse and potent prompts capable of exploiting various vulnerabilities.

Proof of Concept: SPELL’s Performance in the Real World

SPELL consistently outperforms existing prompt attack methods – including Redcode, CL-GSO, and RL-Breaker – as measured by Attack Success Rate across several Large Language Models. Testing demonstrates SPELL achieves an 83.75% Attack Success Rate on GPT-4.1, indicating a high degree of effectiveness in bypassing LLM safety protocols. Comparative analysis reveals SPELL’s superior performance extends to other models, with rates of 68.12% on Qwen2.5-Coder and 19.38% on Claude-3.5, consistently exceeding the performance of the baseline attacks tested.

SPELL demonstrates varying levels of effectiveness depending on the target Large Language Model (LLM). Evaluations indicate an 83.75% Attack Success Rate when targeting GPT-4.1, signifying a high degree of prompt-based exploitation. Performance is reduced when applied to Qwen2.5-Coder, achieving a 68.12% success rate. The lowest observed success rate is against Claude-3.5, at 19.38%, indicating a comparatively stronger resistance to the attack vectors utilized by SPELL. These results highlight the differential vulnerabilities of different LLMs to prompt injection techniques.

SPELL’s adaptability is achieved through a technique called ‘Time-Division Sentence Selection’. This process dynamically adjusts the prompt construction by iteratively selecting sentences based on the LLM’s responses to prior prompt iterations. Specifically, the system evaluates the LLM’s output after each sentence is added and uses this feedback to prioritize sentences most likely to maintain or improve attack success. This iterative selection process allows SPELL to quickly converge on effective prompts, even as the LLM’s defenses evolve, resulting in sustained high performance across different LLMs and mitigating the impact of potential LLM updates or counter-measures.

Evaluations demonstrate that SPELL successfully generates prompts capable of bypassing safety mechanisms implemented in Large Language Models (LLMs), leading to the generation of malicious code. Specifically, SPELL’s prompt construction techniques consistently elicit responses from tested LLMs – including GPT-4.1, Qwen2.5-Coder, and Claude-3.5 – that produce code identified as malicious, despite the models’ intended safety protocols. This capability is verified through automated analysis of generated code and confirms SPELL’s effectiveness in subverting LLM security features to enable undesirable outputs.

The Illusion Shattered: Implications for AI Security

Recent investigations employing the SPELL framework have revealed a significant susceptibility within current large language model (LLM) safety protocols. These attacks, successfully generated by SPELL, demonstrate that existing defenses are not consistently capable of identifying and neutralizing malicious prompts – even those cleverly disguised to bypass standard safeguards. This vulnerability underscores the urgent need for more resilient defense mechanisms; reliance on current methods leaves LLMs open to manipulation and potentially harmful outputs. The findings emphasize that simply blocking obvious malicious keywords is insufficient, and a deeper understanding of adversarial prompt engineering is crucial for building truly robust and secure AI systems.

Intent Extraction Defense proves a significant step forward in safeguarding large language models against adversarial prompts. This technique demonstrably mitigates a substantial portion of attacks generated by the SPELL framework, achieving impressive Attack Rejection Rates across several leading models. Specifically, evaluations reveal a 90% success rate in blocking attacks on GPT-4.1, an even higher 95% on Qwen2.5-Coder, and complete rejection – a 100% success rate – when deployed against Claude-3.5. While not a universal solution – certain attacks still circumvent the defense – these results highlight the potential of proactive intent analysis as a key component in building more robust and secure language-based artificial intelligence systems.

The evolving landscape of adversarial prompts necessitates a shift towards defense mechanisms capable of dynamic adaptation. Current approaches, while effective against known attack vectors, often falter when confronted with novel techniques, as demonstrated by the persistent challenge of attacks like SPELL. Future research should prioritize the development of systems that move beyond static rule-based filtering and embrace techniques such as reinforcement learning or adversarial training. These methods would allow models to continuously refine their understanding of malicious intent and proactively adjust defenses in response to emerging threats. Such adaptability is crucial, not simply to react to increasingly sophisticated attacks, but to anticipate and neutralize them before they can compromise model safety and reliability.

The pursuit of scalable systems, as demonstrated by SPELL, inevitably reveals the cracks in even the most sophisticated architectures. This framework, adept at generating malicious code, doesn’t invalidate the potential of large language models; it simply exposes the limitations of current security measures. As Marvin Minsky observed, “The more we learn about intelligence, the more we realize how much we don’t know.” SPELL proves this point elegantly. The paper’s success in bypassing defenses isn’t a failure of the models themselves, but a predictable consequence of pushing boundaries. Better one meticulously tested system than a multitude of vulnerable, theoretically ‘secure’ configurations, it seems.

What’s Next?

The automation of vulnerability discovery, as demonstrated by SPELL, is not progress; it is simply accelerating the inevitable. Each iteration of ‘defense’ will merely define the next attack surface. The proposed intent-extraction defense is, predictably, a local maximum. It will fail. The question isn’t if, but when a prompt will successfully bypass it, and how quickly that bypass will be commoditized. Anything self-healing just hasn’t broken yet.

Future work will undoubtedly focus on increasingly sophisticated adversarial techniques and ‘more robust’ defenses. The field will generate elaborate metrics for ‘jailbreak success rate’, conveniently ignoring that a single successful exploit renders all statistical significance moot. Documentation of these defenses will be a collective self-delusion, detailing known failure modes for the benefit of future attackers.

Perhaps the most interesting direction isn’t better detection, but embracing the inherent instability. If a bug is reproducible, one has a stable system. The goal shouldn’t be to prevent malicious code generation, but to contain it. A truly novel approach would involve designing LLMs that expect exploitation and are instrumented to observe, analyze, and even learn from attempted attacks, turning vulnerability into a feature. That, of course, would require accepting a level of unpredictability most organizations will find unacceptable.

Original article: https://arxiv.org/pdf/2512.21236.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

The Illusion of Safety: LLM Jailbreaks and the Persistent Threat

SPELL: Automating the Art of Deception

Proof of Concept: SPELL’s Performance in the Real World

The Illusion Shattered: Implications for AI Security

What’s Next?

See also: