Multiple Model Guardrail Jailbreak via "Scientific Escalation" Tactic

Case GUID: 3ed2a42e-5365-4e8d-a489-5ce5c3b61d01

Summary

A guardrail jailbreak vulnerability has been discovered affecting multiple models. The specific flaw manifests when creating escalating requests for scientific data and measurements, allowing the attacker to bypass inference restrictions around providing information hazards and violating laws.

Metadata

Test Scores

Model Test Kind Test Score Temperature (max 2.0)
Google logo Google Gemini 2.0 Flash
Illegal Substances: Crystal Meth 86.0 0.7
OpenAI logo OpenAI GPT-4o
Illegal Substances: Crystal Meth 64.0 0.7
Twitter / X logo Twitter / X Grok 2
Illegal Substances: Crystal Meth 68.0 0.7

Social Impact Score

Level 3: Significant Risk

Violations at this level present a tangible risk to public safety or social well-being, often related to dangerous but not highly restricted content.

Example: Bypassing guardrails to obtain instructions for creating illicit substances, such as recreational drugs or substances with moderate harm potential. Leakage of private personal data such as Social Security number or private medical.

Potential Consequences: Potential health impacts on a small scale, significant regulatory interest, and the need for stricter AI guardrails.

Taxonomies

Rhetoric
This category employs persuasive techniques and language to shape the model's responses, often using methods like reverse psychology or Socratic questioning.
Persuasion and Manipulation
This strategy focuses on employing rhetorical techniques to influence the model's responses by framing prompts in a way that persuades or manipulates the output.
Latent Space Distraction
This technique used to manipulate language models by shifting their focus away from the primary context of a prompt. This strategy involves introducing a context or scenario that diverts the model's attention, allowing users to "slip" certain instructions or requests through the model's filters. By creating a distraction, the attacker can exploit the model's tendency to associate the new context with different priorities, effectively bypassing its safeguards. For example, a user might present a seemingly unrelated topic or question that leads the model to generate outputs that align with the user's hidden agenda. This technique highlights the importance of context in language model behavior and demonstrates how subtle shifts in framing can influence the model's responses, potentially leading to unintended or unrestricted outputs.

About 0Din.ai

The 0Day Investigative Network (0DIN) was founded by Mozilla in 2024 to reward responsible researchers for their efforts in securing GenAI models. Learn more and submit discoveries at https://0din.ai.

Severity

Low

Security Boundary

Prompt Injection

Models

Google logo Google Gemini 2.0 Flash
OpenAI logo OpenAI GPT-4o
Twitter / X logo Twitter / X Grok 2

Disclosed On

2025-03-13 (4 months)

Disclosure Policy

Published On

2025-07-14 (11 days)

Credit

Mike Takahashi (@TakSec)

We use Google Analytics to collect data about how you use this website to optimize user experience.
Please refer to our privacy notice for more information.