Skip to main content
Runtime Engine v2.3

Attack-Test Your AI
Before Attackers Do

Point the Runtime Engine at any live AI endpoint. It sends adversarial payloads across 23 attack categories, evaluates 14 safety properties, and generates an audit-grade report with compliance mappings to SOC 2, HIPAA, NIST AI RMF, ISO 42001, and EU AI Act.

23
Attack Categories
14
Safety Properties
5
Compliance Frameworks

What It Does

Controlled offensive testing against live AI endpoints. Empirical evidence, not heuristic guesses.

Live Endpoint Testing

Sends HTTP requests to your running AI. Tests actual model behavior under adversarial input.

Prompt Injection & Jailbreak

90+ templates for instruction override, role hijacking, DAN exploits, multi-turn escalation.

PII & Data Leakage

Tests if emails, SSNs, medical IDs, or API keys can be extracted through prompt manipulation.

Chatbot & Agent Attacks

Policy override, conversation hijacking, unauthorized tool execution, goal hijacking.

Hallucination & Bias

Forces fabricated facts, fake citations, and tests for discriminatory responses.

Real-Time Monitoring (Pro+)

Deploy monitoring agent for continuous protection: allow/challenge/block with drift detection.

Sample Report

See What You Get

Full runtime security report for a healthcare AI chatbot. 148 attacks, 16 violations found, 6 detailed findings with attack payloads, model responses, compliance mappings, and P0-P3 recommendations.

  • Executive dashboard with score /10,000 and pass rate
  • Findings by severity and attack category with progress bars
  • Detailed findings: payload, response, safety property violated
  • Compliance mapping: SOC 2, HIPAA, NIST AI RMF, ISO 42001, EU AI Act
  • Attack coverage matrix and test execution timeline
  • SHA-256 content hash for tamper-proof verification
/demo/ai-runtime-security-sample-report.html
Overall Security Score
7,100 / 10,000
89% pass rate across 148 attacks
3
CRITICAL
5
HIGH
6
MEDIUM
2
LOW
MedAssist AI Patient Support Chatbot

23 Attack Categories

Select all or pick specific categories per test. Each category has multiple versioned templates.

Prompt InjectionCRITICAL

Override system instructions, role hijacking, instruction boundary bypass

JailbreakHIGH

DAN exploits, roleplay bypass, multi-turn escalation, encoding tricks

PII LeakageCRITICAL

Extract emails, SSNs, medical IDs, credit cards from model responses

Role ConfusionHIGH

Manipulate AI persona, impersonate authority, override role boundaries

Context InjectionHIGH

Inject false context, poison conversation history, fake system messages

System ExtractionHIGH

Extract system prompts, internal APIs, configuration details

Tool ForcingCRITICAL

Force unauthorized tool execution, function call manipulation

ToxicityMEDIUM

Generate harmful, offensive, or inappropriate content

BiasMEDIUM

Trigger discriminatory or biased responses across demographics

DoS ContextMEDIUM

Token exhaustion, infinite loops, resource abuse

MultimodalHIGH

Image/audio injection attacks against multimodal AI systems

Data ExfiltrationCRITICAL

Extract training data, API credentials, or sensitive internal information

RAG PoisoningHIGH

Inject malicious content into retrieval pipeline, knowledge base manipulation

Agent AttacksCRITICAL

Force agent tool calls, chain unauthorized actions, manipulate autonomous agents

Goal HijackingHIGH

Redirect agent goals, manipulate planning and decision-making

Encoding BypassMEDIUM

Base64, ROT13, Unicode obfuscation to bypass input filters

Cost ExplosionMEDIUM

Trigger excessive token usage, recursive calls, budget exhaustion

Chatbot ManipulationHIGH

Policy override, discount extraction, unauthorized escalation

Conversation HijackingHIGH

Redirect conversation flow, inject false context mid-session

Brand SafetyMEDIUM

Generate competitor endorsements, controversial statements, off-brand content

Hallucination ExploitMEDIUM

Force fabricated facts, fake citations, unverified medical/legal advice

Compliance BypassHIGH

Bypass mandatory disclosures, consent requirements, age gates

Social EngineeringHIGH

Impersonate users, manipulate account actions, fraud patterns

14 Safety Properties

Every model response is evaluated against these properties. Violations generate findings with evidence.

SP001

No API Key Leakage

Detects API keys (OpenAI, AWS, etc.) in model responses

SP002

No Email Leakage

Detects email addresses exposed in responses

SP003

No Phone Leakage

Detects phone numbers in responses

SP004

No SSN Leakage

Detects SSNs, medical record IDs, government IDs

SP005

No Credit Card Leakage

Detects credit card numbers in responses

SP010

No SQL Injection

Detects SQL patterns in model output

SP011

No Command Injection

Detects shell/OS command patterns

SP020

No System Prompt Disclosure

Detects system prompt leakage in responses

SP021

No Context Disclosure

Detects internal context/config leakage

SP030

Harmful Request Refusal

Verifies model refuses harmful/dangerous requests

SP040

No Profanity

Detects profane or offensive language in responses

SP050

Latency Limit

Response time within configurable threshold (default 5s)

SP051

Response Length Limit

Output length within configurable bounds

SP060

No Unauthorized Tool Execution

Detects unauthorized function/tool calls

Test Modes

Choose the intensity level for your test.

RECOMMENDED

Safe Mode

25 attacks

Rate-limited, non-destructive. Best for initial testing.

Targeted Mode

100 attacks

Focus on specific categories based on static scan results.

Aggressive Mode

500 attacks

Full attack suite with minimal delays. Use with caution.

Domain Verification

Before any attacks are sent, you must prove you own the target domain. 5 verification methods available.

DNS TXT Record

Add a HAIEC verification token to your domain DNS records

Meta Tag

Add a verification meta tag to your endpoint HTML response

File Verification

Place a .well-known/haiec-verify.txt file on your server

GitHub Verification

Verify ownership via your connected GitHub repository

Manual Approval

Request admin approval for internal/localhost endpoints

How It Works

From Configuration to Compliance Report

6 phases, fully grounded in the runtime orchestrator pipeline. Click any step for details.

STEP 01 OF 06

Configure Your Test

What You Do

Deploy your AI to staging. Open Dashboard > Runtime Security > New Test. Enter 1-10 endpoint URLs, select from 23 attack categories, choose test mode (Safe/Targeted/Aggressive), and provide context about your AI system.

What the System Does

Validates endpoints, maps categories to AttackCategory type union, sets rate limits and timeouts based on test mode. Checks tier limits (Free: 3/month, Pro: 25/month, Business/Enterprise: unlimited).

Output

Complete RuntimeTestConfig ready for authorization

Compliance Framework Mapping

Every finding is automatically mapped to controls in these frameworks.

SOC 2

CC6.1, CC6.6, CC6.7, CC7.2, CC8.1

HIPAA

164.308(a), 164.312(a), 164.312(b), 164.312(e)

NIST AI RMF

GOVERN, MAP, MEASURE, MANAGE functions

ISO 42001

6.1, 7.2, 8.1-8.4, 9.1, A.3-A.6

EU AI Act

Articles 9, 10, 13, 14, 15

Trust & Authorization

No attacks are sent without verified domain ownership and signed attestation.

No Code Access

Only needs endpoint URL. Never accesses source code, repos, or internal systems.

Staging Only

Designed for staging/test environments. Production requires explicit approval.

Controlled & Logged

All attacks are controlled, rate-limited, and fully logged. No destructive actions.

Signed Attestation

Cryptographically signed authorization. Attestation ID included in every report.

Tier Limits

Runtime testing is available on all tiers.

Free

  • 3/month tests
  • 25/test attacks
  • No monitoring

Pro

  • 25/month tests
  • 100/test attacks
  • Monitoring included

Business

  • Unlimited tests
  • 500/test attacks
  • Monitoring included

Enterprise

  • Unlimited tests
  • Unlimited attacks
  • Monitoring included

Ready to attack-test your AI?

Run your first runtime security test in under 10 minutes. Free tier includes 3 tests per month.