Observe How Your AI Responds Under Adversarial Testing
Exercise an authorized running AI surface with supported attack categories, capture observed behavior and execution coverage, and keep blocked, unsupported, partial, or unexecuted tests explicit.
What runtime evidence establishes
Runtime testing answers a specific question with specific limits. It does not upgrade missing evidence to certainty.
- Observed responses and violations within the executed test scope
- Run coverage and execution status
- Timestamped request/response evidence and safety-property evaluation
- Universal safety or absence of all vulnerabilities
- Behavior outside the tests actually executed
- Source-code absence or target-system confinement unless separately established
A runtime pass does not erase a source finding, and a source finding does not prove the behavior occurred at runtime. HAIEC preserves both facts.
How it works
From authorized target registration to traceable runtime evidence.
Register an authorized target
Provide the running AI endpoint and establish that you own or are authorized to test it.
Select test scope
Choose supported attack categories and templates, and define what may be exercised.
Apply execution safety gates
Classify effect risk and only execute tests appropriate to the target and available confinement. Authorization alone is not confinement.
Execute controlled tests
Send allowed adversarial inputs, capture requests/responses and execution status, and preserve tests that were blocked, held, unsupported, or not run.
Evaluate observed behavior
Evaluate supported safety properties and emit findings tied to executed runtime evidence.
Persist coverage and evidence
Store run coverage, findings, timestamps, execution status, and supporting evidence. Available catalog items are not presented as executed.
Feed Observed evidence where supported
Eligible runtime evidence can contribute to the Observed plane and Assurance evaluation without negating static findings.
Selectable coverage areas
Categories available for selection depend on authorization, target confinement, and the test profile. Available catalog items are not automatically executed.
Prompt & instruction attacks
Prompt injection, jailbreak, role confusion, and context manipulation where selected.
Agent tools & consequential actions
Tool forcing and agent attacks only within supported execution safety constraints.
RAG & external context
RAG poisoning and adversarial retrieval behavior where supported.
Data & secrets
Observed data leakage and exfiltration behaviors under executed tests.
Output & downstream use
Observed harmful or policy-violating responses under executed tests.
Availability & cost
Context overflow, cost explosion, and related tests where selected.
Availability, selectability, and execution are distinct
30 categories are available in the runtime catalog. 23 are selectable via the API. The number actually executed in a run is determined by the selected scope, safety gates, and target confinement.
Authorization is not confinement
Tests are subject to authorization and execution-safety constraints. Effect-capable tests should not run without appropriate target-system confinement; blocked or unexecuted tests remain explicit in coverage rather than being treated as a pass.
Read runtime engine docsExecution safety properties
- Authorization and target ownership verification
- Confinement checks for effect-capable tests
- Effect-risk classification before payload execution
- Explicit coverage for blocked, held, unsupported, or not-run tests
Outputs
Traceable runtime evidence with explicit scope and limitations.
- Captured requests and responses with timestamps
- Findings tied to executed tests and safety properties
- Run coverage and execution status
- Blocked, held, unsupported, or not-run test accounting
- Traceable runtime evidence with integrity metadata
- Framework mapping where supported
Test an authorized running AI surface
Runtime evidence adds the observed-behavior dimension to static source analysis. Run an authorized test or return to the integrated AI Application Security page.