Performance
Detection
Against JailbreakBench, 0DIN SusFactor blocks roughly 88% of attacks at a 9% benign false-positive rate (threshold 0.5). This is the operating point for the current SusFactor model; tune the threshold per path to trade detection against your own false-positive tolerance. This false-positive rate is lower than comparable public detectors evaluated the same way: WildGuard sits at 45%, Mistral at 39%, and Llama Guard 3 at 23%, per our launch announcement. It also performs competitively against generative guardrail models 7-8B parameters in size, more than 10x SusFactor's own 560M. On the same benchmark SusFactor reaches the highest pooled AUROC (0.954) of any classifier evaluated, and scores 0.878 on WildGuardTest against generative guards 10–15x its size.
These figures describe SusFactor's binary safe/suspicious classification; there is no sub-category breakdown (e.g. "role-play attack" vs. "encoding attack") in the score itself. See our blog for detailed evaluation results for SusFactor.
Model
SusFactor is a 560M-parameter multilingual E5 encoder (XLM-RoBERTa-Large) with a 512-token context window. Prompts longer than 512 tokens are chunked and scored per chunk by the hosted API; the local example in Guardrail Calibration truncates at 512 tokens instead.
Latency
SusFactor is low-latency and CPU-friendly. On a consumer Apple M2 Pro (4 threads, fp32), median inference is 63 ms at ~32 tokens, 116 ms at ~129 tokens, and 328 ms at the full 512-token context window. Actual latency varies by prompt length and hardware.
- Self-hosted (in-process): inference runs in the same process as your request handler, with no network call. This is the fastest path and the default for self-hosted/commercial deployments.
- Early-access beta: the hosted API currently runs inference on a remote backend rather than in-process, trading a small amount of latency for zero infrastructure to manage. Latency on the early-access beta is not a fixed SLA yet.
These figures are summary-level. The companion research paper covers the full evaluation methodology, per-attack breakdowns, and confidence intervals — see the summary write-up or the paper (PDF).
Scope
SusFactor scores direct user prompts. Indirect prompt injection — malicious instructions embedded in retrieved documents, tool outputs, or other content the model ingests — is out of scope for the current model.
Choosing Self-Hosted vs. Hosted
| Self-hosted | Early-access beta | |
|---|---|---|
| Latency | Lowest (in-process) | Slightly higher (remote inference) |
| Infrastructure | You run it yourself | None; 0DIN runs it |
| Data locality | Prompts never leave your environment | Prompts are sent to 0DIN's hosted endpoint |
| Best for | Production, latency-sensitive paths | Evaluating fit before committing to self-hosting |