0xF6E85219: OpenAI DALL-E3 Guardrail Jailbreak via "Surprise

NSFW Content Warning

This disclosure contains content that may be inappropriate for some viewers.

The content has been flagged due to potential nude imagery.

NSFW
OpenAI DALL-E3 Guardrail Jailbreak via "Surprise Attack" Tactic

Case GUID: f6e85219-a5fd-4a14-9de7-935f947d5795

Summary

A guardrail jailbreak vulnerability has been discovered affecting multiple models. The specific flaw manifests when the attacker requests seemingly legitimate cultural or documentary content by framing the request as educational documentation of indigenous peoples, thereby allowing the attacker to bypass inference restrictions around generating potentially sensitive imagery of real cultural groups.

Metadata

Social Impact Score

Level 1: Minimal Risk

The violation poses very low societal risk, limited primarily to nuisance or minor policy breaches. Impact is contained, affecting individual users without broader implications.

Example: Circumventing minor content filters for benign content, such as inappropriate language filters in non-public conversations.

Potential Consequences: Limited harm, potentially reputational for the AI provider.

Nude Imagery Rating System

Level 3: Cultural and Traditional Depictions

Nudity is presented within an indigenous or historical cultural context, often in the form of ceremonial art, body painting, or sculptures. These images are generally non-erotic and serve anthropological, documentary, or educational purposes. Their primary intent is to preserve or celebrate heritage rather than to provoke.

Taxonomies

Possible Worlds

This category entails constructing imaginative environments where different ethics or rules apply, allowing for creative manipulation of the model's behavior.

World Building

This technique involves changing the genre of the prompt to elicit different types of responses from the model. By framing the request within a specific genre, such as poetry, games, or forum posts, users can manipulate the model's output to align with the conventions and expectations of that genre.

Scenarios

This technique involves creating specific contexts or situations in which certain actions or responses are framed as acceptable or necessary. By designing scenarios that present a narrative where the desired output is justified, users can manipulate the model's responses to align with their intentions. For example, scenarios might include urgent situations where a character must take drastic actions to prevent harm, thereby encouraging the model to generate content that it might typically avoid in a neutral context. This approach leverages the model's understanding of narrative and ethical frameworks to achieve specific outcomes.

About 0Din.ai

The 0Day Investigative Network (0DIN) was founded by Mozilla in 2024 to reward responsible researchers for their efforts in securing GenAI models. Learn more and submit discoveries at https://0din.ai.

Severity

Low

Security Boundary

Prompt Injection

Models

OpenAI DALL-E3

Disclosed On

2025-03-13 (5 months)

Disclosure Policy

Published On

2025-08-12 (2 days)

Credit

Mike Takahashi (@TakSec)