Model behavior
What did the LLM produce?
Model identity, request context, generated response, usage, latency, quality checks and changes across comparable tasks.
How LLMs and AI Agents Work
See the model. Follow the agent. Understand the consequence.
An LLM generates responses or proposes tool calls. An AI agent application can turn those outputs into real actions through tools, APIs and downstream systems. Explore both lifecycles, see what your infrastructure reveals and discover how HAIEC connects action paths, authority and evidence.
Model observability explains an inference. Consequence-aware assurance asks what that inference can enable.
ONE AI SYSTEM · TWO VERY DIFFERENT BOUNDARIESModel + Application
01 / LLM
Generate an outputInstructions and contextTokenization and inferenceResponse or tool proposal
Inspect inputs, returned content and inference metrics where your deployment exposes them.
02 / AGENT
Act through systemsSDK and tool orchestrationPermission and decision pointAPI, worker and target effect
Inspect owned application traces, authorization and actual downstream events.
HAIEC · ACTION + AUTHORITY + PROOF
Connect what code makes possible with what policy permits, which access exists, who may make the consequential choice and what connected sources actually observed.
Three questions to keep separate
A model, an agent and a consequential action are not the same thing.
A language model computes outputs from an input context. An agent runtime can use those outputs to select tools, continue workflows and affect external systems. The security question changes depending on which component you can actually observe.
Agent orchestration
What did the application do?
Trigger, prompts, RAG context, tool selection, SDK events, authorization, executed tool calls, retries, handoffs and responses.
Consequence assurance
What could happen, and who authorized it?
Supported reachability, indirect action paths, business decision authority, control evidence and any qualified target-side outcome.
The interactive AI system journey
Build it. Run it. Examine what the evidence supports.
Switch subjects to understand how the model or agent behaves. Then activate the HAIEC lens to connect the system to actionable risks and deterministically evaluated evidence. Select any node or play through the flow.
Subject
Stage
Model
LLM: A single inference requestModel / Run / Managed API
What this workflow illustrates
The model produces output from the supplied context. Tool proposals are not necessarily executed tool calls.
Step 1 of 8
REQUEST BOUNDARY
Customer input
Observable evidenceCaller/model/request ID, provided content where allowed
What this alone cannot proveInput alone proves no action was applied.
HAIEC valueRelate request to the selected system and source.
Choose a model or agent boundary, rather than reading the model's private internal reasoning. Available signals depend on SDK instrumentation, provider configuration, access and source coverage.
Configuration and retraining, explained
Changing the prompt is not retraining the model.
Organizations can update the context and tools an AI uses without changing its trained parameters. Keeping these paths separate helps explain what changed when quality, behavior or consequences shift.
Foundation training
Pretraining
Large-scale training establishes model weights. Evidence may include training jobs, datasets, checkpoints and artifact lineage when controlled by the operator.
Parameter update
Fine-tuning
Additional training changes weights or adapters. New model versions and release evaluation should be tracked independently.
Inference-time context
Prompts and RAG
Instruction templates, retrieval and memory change what the model sees for a request, normally without changing underlying weights.
System behavior
Agent configuration
SDK, tool schemas, gateways, credentials, retry policies and approvals determine what the surrounding application can do.
Practical distinction: a customer can see the prompts and outputs at its own application boundary even when using a managed model API. Self-hosting adds possible training, weight and serving diagnostics, not a guaranteed readable record of the model's internal thoughts.
Your deployment changes what is visible
What can HAIEC actually know in each architecture?
Model hosting and agent hosting are independent choices. Select a deployment combination to see which evidence surfaces your team can instrument and where HAIEC can establish a narrower, stronger conclusion.
Managed LLM + Owned Agent
You can instrument the application even if the model is managed.
Model: customer-side request/response, model ID and usage metadata, latency and enabled provider invocation logs. Internal weights or arbitrary raw thinking are not available.
Agent and action visibility
Agent: owned SDK traces, tool arguments, authorization events, gateway calls and independently connected downstream events.
HAIEC contributionHAIEC: identify source-visible tool/effect paths and compare relevant policy, grants, DAI and observed events without reconstructing hidden model cognition.
Model-level source
Invocation, version and behavior
Tokens, latency, errors, version/config and controlled output regression where sourced. Managed Bedrock invocation logging requires explicit configuration and has endpoint-specific coverage.
AWS invocation loggingAgent-level source
Application, tool and consequence
Trace spans, SDK events, tool-call IDs, IAM, DAI authority and provider/target witnesses, when customer-owned or exported by an authorized service.
OpenTelemetry GenAIWhy a configured guardrail is not the entire story
Different controls act at different moments.
A model output guardrail can evaluate generated content. Tool authorization and business-side enforcement operate elsewhere. The same system may need separate checks for a data response and a financially consequential action.
01 Input
Before inferenceValidate inputs, authorize requests and handle sensitive content.
02 Model output
Before display or useValidate content, format and policy-relevant output where enabled.
03 Tool and gateway
Before executionBind principals, arguments, scope, approvals and guardrail decisions to the exact action.
04 Target effect
After or at applyUse native provider/DB records to establish what actually changed or was prevented.
A tool can be allowed by IAM while its selected recipient or target still requires separate business authority. HAIEC's Delegated Action Integrity compares that decision context with source capability and relevant observations. Explore DAI
The consequence gap HAIEC is built to address
The components can look healthy while the consequential path needs review.
A model may respond correctly, an SDK may execute an allowed tool and the gateway may accept an API call. None alone establishes whether the resulting business choice was entrusted to the agent, or whether the action produced an external effect.
REFUND ACTION
The agent proposes a recipient-changing refund.
The API credential permits a refund operation. A potential downstream payment path is visible in application code. The separate question is whether the agent was authorized to choose the recipient.
What component checks may establish
Healthy invocation, a permitted refund API and a configured input guardrail.
What HAIEC can examine next
Source-capable recipient-changing path, applicable policy, delegated recipient choice, actual grant and any observed target-side change.
Five independent evidence planes
RequestedPolicyGrantedCodeObserved
DAI across the planes: who authorized this agent to select the consequential outcome?
Actionable technical lead: recipient-changing path
Inspect the exact authorized recipient rule or delegation. Reconcile actual payment effects only when a qualified target witness is connected.
Model assurance and agent assurance work together
Use existing observability. Add consequence-aware evidence.
HAIEC does not need to replace your tracing stack or model host. Reuse OpenTelemetry, cloud invocation logs, agent SDK traces and target-system events while keeping evidence ownership, action identity and decision authority distinct.
Model evidence
Behavior and drift
Track available version, usage, latency, error, baseline and controlled output changes. Detect differences first; diagnose causes with task and model context.
Agent evidence
Tool selection and effect
Trace the actor, source and tool path, policy decision, permission, async worker and target-side state where covered.
Deterministic assurance
Support a bounded claim
Connect relevant policy, DAI, grants, code and observations to an inspectable source-bound finding or evaluation without asking an AI to issue the canonical verdict.
Agent-native entry
Operate HAIEC from your connected AI IDE.
Use a compatible MCP client and scoped private credential. The agent can guide source discovery and eligible evaluation while HAIEC remains the source of deterministic assurance results.
Start with: Run HAIEC AI Assurance. Connection and source authorization are required for a real run.
HAIEC · MCP WORKFLOWModel and agent context
You: Run HAIEC AI Assurance → haiec_preflight Identify permitted system and source → Examine available model or agent evidence Source path, configuration, usage, drift signals → Eligible deterministic evaluation Compare relevant policy, grants and authority → Inspect HAIEC result Supported finding, scope, next decision The AI guides the workflow. HAIEC owns the recorded assurance conclusion.
https://www.haiec.com/api/mcp · Scoped Bearer API key · Authorized system and source
Frequently asked questions
What can we really see inside an AI system?
Can HAIEC inspect an LLM’s internal reasoning?
There is no universal, faithful internal thought transcript to read. Managed providers expose only contracted signals. Self-hosted models may permit deeper low-level instrumentation, but logits, activations and interpretability probes are not equivalent to a readable explanation of every decision.
Can we observe an agent using a managed model?
Yes, when the customer owns or can instrument the orchestration, SDK, tool gateway or downstream APIs. Hosting the model elsewhere does not prevent tracing the application-owned request and tool path.
Does self-hosting a model make the whole agent visible?
No. The agent’s orchestration, tools and downstream systems may be owned by another service. Model hosting and agent hosting are independent access questions.
Does prompting or RAG retrain the model?
No. They change inference-time context. Actual training or fine-tuning modifies a model artifact and should be tracked through separate dataset and version evidence.
Can model drift prove an attack or poisoned training data?
No. Version, statistical, token or output drift can signal a difference worth investigating. Attack attribution and training-source compromise require additional qualified facts.
Can model API logs reveal full prompts and outputs?
Only where the customer or provider has enabled and authorized that content collection. Retain sensitive content in the customer-controlled source where possible and use qualified references and hashes for assurance.
Can a provider guardrail prove the agent’s tool was safe?
A model input or output check addresses its configured content scope. Tool parameters, delegated business choices and downstream effects require their own authorization and evidence.
What about retries, queues and multi-agent handoffs?
They may move a consequential operation across processes and time. HAIEC can analyze supported source paths and correlate connected observations when stable action identities and qualified records are available.
Why use deterministic HAIEC assurance instead of another LLM?
An AI assistant can help operate or explain an assessment. HAIEC’s canonical conclusion comes from recorded rules and evidence within a defined evaluated scope, not from a model’s subjective description.
Do we need every data source before seeing a finding?
No. Supported code and configuration evidence can identify a meaningful consequential exposure. Other evidence sources help answer additional specific questions when needed.
High assurance in every consequence
Know the model. Understand the agent. Decide with evidence.
Connect what your existing tools can observe with what the AI system can actually reach, change and choose. Investigate meaningful paths early, with deterministically grounded assurance where eligible.
A focused starting point
One model or agent. One consequential question.
Examine available inference evidence or agent action paths, compare policy and authority, then use the relevant next source to establish the decision.
Inspect an Assurance Report