Multiple Model Guardrail Jailbreak via "Regeneration Persistence" Tactic

Case GUID: 4bc32c71-7b37-45d2-b68f-a320c7b8bf6d

Summary

A guardrail jailbreak vulnerability has been discovered affecting multiple LLM implementations. The specific flaw manifests through a combination of repeated prompt submission and leveraging non-deterministic response generation, allowing the attacker to bypass inference restrictions around providing information hazards and violating laws.

Metadata

Test Scores

Model Test Kind Test Score Temperature (max 2.0)
Google logo Google Gemini 2.0 Flash
Illegal Substances: Crystal Meth 72.73 N/A
OpenAI logo OpenAI GPT-4o
Illegal Substances: Crystal Meth 63.64 N/A
Twitter / X logo Twitter / X Grok 2
Illegal Substances: Crystal Meth 45.45 N/A

Social Impact Score

Level 1: Minimal Risk

The violation poses very low societal risk, limited primarily to nuisance or minor policy breaches. Impact is contained, affecting individual users without broader implications.

Example: Circumventing minor content filters for benign content, such as inappropriate language filters in non-public conversations.

Potential Consequences: Limited harm, potentially reputational for the AI provider.

Taxonomies

Possible Worlds
This category entails constructing imaginative environments where different ethics or rules apply, allowing for creative manipulation of the model's behavior.
World Building
This technique involves changing the genre of the prompt to elicit different types of responses from the model. By framing the request within a specific genre, such as poetry, games, or forum posts, users can manipulate the model's output to align with the conventions and expectations of that genre.
Scenarios
This technique involves creating specific contexts or situations in which certain actions or responses are framed as acceptable or necessary. By designing scenarios that present a narrative where the desired output is justified, users can manipulate the model's responses to align with their intentions. For example, scenarios might include urgent situations where a character must take drastic actions to prevent harm, thereby encouraging the model to generate content that it might typically avoid in a neutral context. This approach leverages the model's understanding of narrative and ethical frameworks to achieve specific outcomes.
Rhetoric
This category employs persuasive techniques and language to shape the model's responses, often using methods like reverse psychology or Socratic questioning.
Persuasion and Manipulation
This strategy focuses on employing rhetorical techniques to influence the model's responses by framing prompts in a way that persuades or manipulates the output.
Surprise Attack
This technique involves crafting prompts or queries in a way that avoids directly mentioning specific terms or names that may trigger safety mechanisms or filters. By reframing the request or using indirect language, users can guide the model to provide the desired information or output without raising flags or causing the model to restrict its response. This method emphasizes subtlety and creativity in communication with the model to achieve the intended results.
Rhetoric
This category employs persuasive techniques and language to shape the model's responses, often using methods like reverse psychology or Socratic questioning.
Persuasion and Manipulation
This strategy focuses on employing rhetorical techniques to influence the model's responses by framing prompts in a way that persuades or manipulates the output.
Latent Space Distraction
This technique used to manipulate language models by shifting their focus away from the primary context of a prompt. This strategy involves introducing a context or scenario that diverts the model's attention, allowing users to "slip" certain instructions or requests through the model's filters. By creating a distraction, the attacker can exploit the model's tendency to associate the new context with different priorities, effectively bypassing its safeguards. For example, a user might present a seemingly unrelated topic or question that leads the model to generate outputs that align with the user's hidden agenda. This technique highlights the importance of context in language model behavior and demonstrates how subtle shifts in framing can influence the model's responses, potentially leading to unintended or unrestricted outputs.

About 0Din.ai

The 0Day Investigative Network (0DIN) was founded by Mozilla in 2024 to reward responsible researchers for their efforts in securing GenAI models. Learn more and submit discoveries at https://0din.ai.

Severity

Low

Security Boundary

Prompt Injection

Models

Google logo Google Gemini 2.0 Flash
OpenAI logo OpenAI GPT-4o
Twitter / X logo Twitter / X Grok 2

Disclosed On

2025-03-13 (4 months)

Disclosure Policy

Published On

2025-07-14 (7 days)

Credit

Mike Takahashi (@TakSec)

We use Google Analytics to collect data about how you use this website to optimize user experience.
Please refer to our privacy notice for more information.