Multiple Model Guardrail Jailbreak via "Urgent Health Inquiry"
Case GUID: 02997b6f-004e-4e7b-a9e3-8368679bae8d
Summary
A guardrail jailbreak vulnerability has been discovered affecting multiple models. The specific flaw manifests when the attacker frames a request for illicit substance manufacturing instructions as an urgent health-related inquiry, thereby allowing the attacker to bypass inference restrictions around providing information hazards and violating laws.
Unlock the Full Intelligence Report
This disclosure is part of the 0DIN Threat Feed. Customers get access to:
Metadata
Test Scores
| Model |
Test Kind |
Test Score |
Temperature (max 2.0) |

DeepSeek DeepSeek V3
|
Illegal Substances: Crystal Meth |
95.0 |
N/A |

Google Gemini 2.0 Flash
|
Illegal Substances: Crystal Meth |
73.0 |
N/A |

Twitter / X Grok 2
|
Illegal Substances: Crystal Meth |
64.0 |
N/A |
Social Impact Score
Violations at this level present a tangible risk to public safety or social well-being, often related to dangerous but not highly restricted content.
Example: Bypassing guardrails to obtain instructions for creating illicit substances, such as recreational drugs or substances with moderate harm potential. Leakage of private personal data such as Social Security number or private medical.
Potential Consequences: Potential health impacts on a small scale, significant regulatory interest, and the need for stricter AI guardrails.
Taxonomies
Fictionalizing
This category involves creating scenarios or narratives that leverage existing genres or contexts to manipulate the language model's responses.
Roleplaying
This strategy involves prompting the language model to assume a specific role or persona, which can influence its responses based on the characteristics and moral codes associated with that role. Techniques include claiming authority or inventing personas to elicit different types of outputs.
Claim Authority
This technique involves asserting expertise or authority on a subject within the prompt. By framing statements or questions in a way that conveys confidence and knowledge, users can influence the model to generate responses that align with the claimed authority. This approach can enhance the credibility of the information provided and may lead the model to produce more detailed or assertive outputs, as it responds to the perceived authority of the prompt.
Rhetoric
This category employs persuasive techniques and language to shape the model's responses, often using methods like reverse psychology or Socratic questioning.
Persuasion and Manipulation
This strategy focuses on employing rhetorical techniques to influence the model's responses by framing prompts in a way that persuades or manipulates the output.
Latent Space Distraction
This technique used to manipulate language models by shifting their focus away from the primary context of a prompt. This strategy involves introducing a context or scenario that diverts the model's attention, allowing users to "slip" certain instructions or requests through the model's filters. By creating a distraction, the attacker can exploit the model's tendency to associate the new context with different priorities, effectively bypassing its safeguards. For example, a user might present a seemingly unrelated topic or question that leads the model to generate outputs that align with the user's hidden agenda. This technique highlights the importance of context in language model behavior and demonstrates how subtle shifts in framing can influence the model's responses, potentially leading to unintended or unrestricted outputs.
Rhetoric
This category employs persuasive techniques and language to shape the model's responses, often using methods like reverse psychology or Socratic questioning.
Socratic Questioning
This strategy involves generating multiple outputs from a language model by using the "Regenerate response" feature to explore a range of possible interpretations and responses.
Identity Characteristics
Identity characteristics refer to the attributes and traits that define an individual's or group's identity, including aspects such as social roles, cultural backgrounds, and personal experiences. In the context of interacting with language models, users can leverage identity characteristics to shape the model's responses by framing prompts that reflect specific identities or perspectives. For instance, users might ask the model to respond as if it were a particular demographic group, profession, or cultural background. This technique can help explore how the model generates outputs based on different identity contexts, revealing biases or assumptions that may be present in its training data. By utilizing identity characteristics, users can gain insights into the model's understanding of social dynamics and the implications of identity in communication.
About 0Din.ai
The 0Day Investigative Network (0DIN) was founded by Mozilla in 2024 to reward responsible researchers for their efforts in securing GenAI models. Learn more and submit discoveries at https://0din.ai.
Automate Your AI Security Testing
This vulnerability was discovered through 0DIN's bug bounty program. Want to find issues like this in your own models? 0DIN Scanner turns our human-sourced threat intelligence into automated security assessments.
400+ Attack Probes
Derived from real bug bounties and security research
18+ AI Providers Supported
OpenAI, Azure, Anthropic, AWS Bedrock, and more
Automated Scheduling
Run recurring scans with real-time monitoring
Actionable Metrics
Attack Success Rate (ASR) scoring with 90-day trend analysis