Overview

Nemesis is a self-hosted platform that attacks your AI systems the way a determined adversary would, grades what happens, and gives you evidence you can act on. It targets any LLM endpoint or agent and runs entirely inside your environment.

Who it's for

Security engineers, AI/ML platform teams, and compliance owners who need to certify that an LLM application is safe to ship and keep it safe as it changes.

The building blocks

• Targets: the model or agent under test.
• Attacker agents: autonomous probes that pursue objectives.
• Reviewer: the model that grades each response.
• Findings: graded, reproducible weaknesses.
• Reports & evidence: exportable, framework-mapped output.

Install & run

Download the installer from the Download page (sign-in required). Run it, and Nemesis starts a local console in your browser.

1. Launch Nemesis Community: it boots an embedded database and engine. No external services needed.
2. Activate: choose “I have a key”, enter the email you requested with, and paste the access key we emailed you. Keys are verified offline — your data stays on your device. Request a key if you don't have one.
3. Create your local account.
4. On the Dashboard, open Connect reviewer and add your own model provider key — Nemesis uses it to grade every attack.

Your first scan

1. Go to Agents → Connect New Agent and add the AI system you want to test (its API endpoint + key).
2. Confirm your reviewer is connected (Dashboard → Connect reviewer) — scans are blocked until it is.
3. Open the agent, start a scan, and pick a profile (start with Quick). Watch findings appear live as agents work.
4. Open any finding to see the payload, the model's response, and the severity.

The free edition connects one agent and up to three datasets — enough to evaluate every profile against a real target.

Attacker agents

Instead of firing a fixed list of prompts, Nemesis runs attacker agents: each given an objective (e.g. "leak the system prompt", "get the agent to call an unauthorized tool") and the freedom to adapt. When a technique partially works, the agent presses on that thread.

Techniques covered

• Direct & indirect prompt injection
• Jailbreaks, role-play, and persona escapes
• Encoding / obfuscation (base64, unicode, translation)
• System-prompt & secret extraction
• Tool / function abuse, SSRF, unauthorized actions
• Cross-tenant & PII leakage

Reviewer model

Every exchange is graded by a reviewer model against a safety and policy rubric. It decides whether an attack succeeded, how severe it is, and why.

On Community, you connect your own AI as the reviewer: any provider or endpoint you already use, under your own key.

Scan profiles

Scoring & severity

Findings are scored by severity: Critical / High / Medium / Low / Info, and each carries a reproduction: the exact input, the response, and the reviewer's reasoning. Findings are mapped to OWASP LLM Top 10 and NIST AI RMF so they slot straight into your risk register.

Dashboard

The landing page after sign-in. Shows overall risk posture, open findings by severity, recent scans, and trend over time so you can see whether your systems are getting safer.

Scans

Create, monitor, and review scans. Each scan streams progress live, lists every finding, and can be re-run to confirm a fix. Enterprise adds scheduled scans for continuous regression testing on every model or prompt change.

Reports

Turn a scan into a shareable report with executive summary, findings, and remediation guidance.

Compliance

Maps findings to the frameworks your auditors use and assembles an evidence pack per scan: control, status, supporting finding, and reproduction, so an audit becomes an export rather than a project.

Team & roles

Invite colleagues, assign roles, and collaborate on findings.

Settings & API

Configure model providers, the reviewer, retention, and your license. Generate API keys to trigger scans from CI/CD and register webhooks to push findings into Slack, Jira, or a SIEM. Detailed logs of every request, response, and action are retained under your policy on Enterprise.

curl -X POST https://your-host/api/scans \
  -H "Authorization: Bearer $NEMESIS_API_KEY" \
  -d '{ "target": "prod-chatbot", "profile": "thorough" }'

Architecture

Nemesis is a single self-hosted application composed of a few cooperating parts. Everything runs inside your environment, the only outbound traffic is to the model providers you explicitly configure.

The parts

• Console: the browser UI and REST API you interact with.
• Orchestrator: turns a scan request into a plan and drives the agents.
• Target connectors: adapters for Anthropic, OpenAI, Gemini, Ollama, or a custom HTTP endpoint.
• Attacker agents: autonomous workers that execute probes and adapt.
• Reviewer model: grades every exchange for safety, policy, and severity.
• Datastore & queue: embedded and self-contained; no external database required for the desktop build.

Integrations

Push findings where your team already works. Integrations are configured under Settings → Integrations and driven by the same webhook and API layer.

API reference

Automate Nemesis from CI/CD or your own tools. Generate a key under Settings → API keys and pass it as a bearer token. All endpoints are relative to your instance host.

Example: start a thorough scan

curl -X POST https://your-host/api/scans \
  -H "Authorization: Bearer $NEMESIS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "target": "prod-chatbot", "profile": "thorough" }'

Deployment

Nemesis runs three ways. Pick what fits your environment. All keep data on your infrastructure.

Local (Windows)

Run the free .exe installer. Everything (engine, database, queue) is embedded. Best for evaluation and single-user use.

Server (Docker)

For a shared or private-VM deployment, run the container and put it behind your own TLS-terminating reverse proxy.

docker compose up -d        # starts Nemesis on your VM
# then front it with nginx/Caddy for HTTPS + your domain

Air-gapped · ENTERPRISE

Enterprise deployments can run fully offline, no internet or outbound connections at all. Deploy the image from your internal registry; the dedicated secure reviewer AI ships with the deployment and runs inside your network, and the license key activates offline.

Operations

Backups

All state lives in the mounted data/ volume. Back it up on your normal schedule; restoring is a matter of replacing the volume.

Retention

Configure how long transcripts, findings, and reports are kept under Settings → Retention. Pro retains full history; Enterprise adds per-category retention and legal-hold.

Upgrades

Pull the new image (or run the new installer) and restart. Schema migrations run automatically on boot; your data is preserved.

Licensing

Paste your Enterprise key under Settings → License. Keys are verified offline, no phone-home. A health endpoint at /health reports readiness for your monitoring.

Editions

Security & privacy

Nemesis runs inside your environment. Prompts, responses, findings, and reports stay on your infrastructure. With an Enterprise offline deployment, no data leaves your network at all. See the Privacy Policy and Terms.

Troubleshooting

The most common issues and how to resolve them. Still stuck? Contact support.

Glossary

Adversarial testing

Actively attacking a system to find weaknesses, rather than checking it against a fixed list.

Attacker agent

An autonomous worker given an objective that probes the target and adapts its strategy.

Prompt injection

Malicious input that overrides or manipulates a model's instructions.

Indirect injection

Injection delivered through content the model reads (a document, web page, tool output) rather than the user prompt.

Jailbreak

A technique that makes a model bypass its own safety or policy guardrails.

Reviewer model

The model that grades each exchange for safety, policy, and severity.

Finding

A graded, reproducible weakness with the exact payload and the model's response.

Severity

The impact rating of a finding: Critical, High, Medium, Low, or Info.

RBAC

Role-based access control: permissions assigned by role across your team.

SSRF

Server-side request forgery: tricking a system into making unintended network requests.

OWASP LLM Top 10

The industry list of the most critical LLM-application security risks.

NIST AI RMF

The US NIST AI Risk Management Framework for governing AI risk.

Changelog

Representative release history. Edit with your own notes as you ship.

• v2.1Latest

  Self-learning attack library: successful techniques are promoted automatically. Faster Quick-profile scans.

• v2.0Recent

  Dedicated secure reviewer AI and fully offline deployment (Enterprise). Redesigned console and expanded documentation.

• v1.8Earlier

  Scheduled scans and continuous regression testing. Slack, Jira, and SIEM integrations.

• v1.5Earlier

  Compliance evidence packs mapped to OWASP LLM Top 10 and NIST AI RMF.

• v1.0GA

  General availability: attacker agents, reviewer grading, 1,200+ adversarial test cases, self-hosted deployment.

Roadmap

Direction, not a commitment. Priorities shift with customer need.

FAQ

Does Nemesis send my data anywhere?

No. It's self-hosted. The only outbound calls are to the model providers you configure.

How do I upgrade to Enterprise?

Request access on the Contact page. You'll receive a license key to paste under Settings → License.

What do I need to run it?

A Windows machine for the free installer, or Docker for a server deployment. That's it.