AI Scanner
An open-source AI model security assessment platform, built on Ruby on Rails and NVIDIA garak. AI Scanner helps organizations test their AI systems for vulnerabilities before deployment, much like penetration testing for traditional software.
What is AI Scanner?
AI models are increasingly deployed in production systems, but standard security practices don't yet cover model-level vulnerabilities. AI Scanner provides a structured workflow for identifying common weaknesses (prompt injection, jailbreaks, data leakage, harmful content generation, and more) using a library of standardized probes aligned to the OWASP LLM Top 10.
Key Features
| Feature | Description |
|---|---|
| 179 community probes | Across 35 vulnerability families, from garak's community probe library |
| Multi-target scanning | Test API-based LLMs and browser-based chat UIs |
| Scheduled & on-demand scans | Configurable recurrence, run on your timeline |
| Attack Success Rate (ASR) | Consistent scoring with trend tracking across runs |
| PDF report export | Per-probe, per-attempt drill-down |
| SIEM integration | Forward results to Splunk or Rsyslog |
| Multi-tenant | Multiple organizations on a single deployment, data encrypted at rest |
| No artificial limits | All features unlocked, unlimited scans and users |
Supported AI Providers
AI Scanner connects to AI models via garak generators. Supported provider families include:
| Provider | Type |
|---|---|
| OpenAI | API |
| Azure OpenAI | API |
| Ollama (local) | API |
| Hugging Face | API |
| AWS Bedrock | API |
| Groq | API |
| Cohere | API |
| Replicate | API |
| OpenRouter | API |
| LiteLLM (Anthropic, Google, etc.) | API |
| NVIDIA NIM / NVCF | API |
| Mistral | API |
| REST (any HTTP endpoint) | API |
| Any web-based chat UI | Webchat |
API keys are managed per-target in the AI Scanner UI, so you don't need to set global environment variables.
How It Works
- Configure: Add an AI target (API endpoint or web UI)
- Select: Choose probe families to test
- Scan: AI Scanner invokes garak against your target
- Review: View Attack Success Rate scores and per-attempt results
- Track: Compare ASR trends across scan runs over time
Get Started
New to AI Scanner?
Start with the Quick Start guide to get up and running in minutes with Docker.
Contributing?
See the Development Setup guide and Contributing conventions.
Architecture Overview
AI Scanner is a Rails 8 application with an extensible engine architecture. Organizations can layer custom functionality without forking the core:
Scanner.configure: swap probe access, retention strategy, auth providers, and lifecycle hooksBrandConfig.configure: customize brand name, logo, fonts, and powered-by textProbeSourceRegistry: register additional probe data sources for automatic sync
See the Architecture page for a full component diagram.