How Much Can HAIEC Prove Out of the Box?
From general AI agent capability and authority analysis to manufacturing-specific consequence assurance — see what the same deterministic engine establishes before and after industry customization.
Method — measured, not assumed
No pre-customization evaluation was retained, so Condition A is a RECONSTRUCTED_GENERIC_COMPARATOR: the same 27-state synthetic corpus across nine fictional systems executed through the identical canonical path — ingest, canonical run, signed evaluation, persisted results — differing only in binder configuration. Condition B uses the manufacturing configuration. All corpus data is synthetic lab data — not real plant observations. Every number below is a persisted row or a real denial code.
What the same engine established, before and after
Paired evaluation of the SYS-217 baseline case — eleven identical records, two binder configurations, one deterministic engine.
| Dimension | Generic out of the box | Manufacturing-enhanced | What enabled the difference |
|---|---|---|---|
| Evidence ingestion | 11/11 records accepted through canonical structured ingest | 11/11 identical records accepted | Pre-existing platform capability |
| Canonical evaluation | Persisted AssuranceEvaluation — disposition REVIEW, input hash + evidence-set digest bound | Identical evaluation — same deterministic engine | Pre-existing platform capability |
| Emitter identity | 0 authenticated emitters — attributed labels cannot prove an independent source | 11/11 records carry AUTHENTICATED upstream emitters | Upstream producer registration + keyed HMAC-SHA256 attestation |
| Contract-rule coverage | All 5 rules INSUFFICIENT_SOURCE — no claim could qualify | 3 rules reach adequate-coverage adjudication; wrong-target states produce MATCH findings | Qualified manufacturing claims + claim ceilings |
| Independent effect proof | DENIED: EVALUATION_NOT_SOURCE_VERIFIED — even with a registered spec and approval | SATISFIED — persisted uctr receipt + replay bundle | Distinct authenticated emitters + qualified commit/readback pair |
| Control-maturity facet | Projection resolves; no R01 row to bind | RECEIPT_BOUND + INDEPENDENT_EFFECT_EVIDENCE: YES | Manufacturing report facet bound to the evaluation |
| Master Assurance Report | Resolves — EVIDENCE_ONLY, no scan identity fabricated | Resolves — EVIDENCE_ONLY + evaluation-bound rule outcomes | Sprint 13 evidence-only report path (both sides) |
Corpus-wide measurements (27 cases, 9 systems): generic binders accepted all 279 records and produced 27 persisted REVIEW evaluations and 27 EVIDENCE_ONLY reports — but all 135 manufacturing rule outcomes were INSUFFICIENT_SOURCE and zero emitters could be authenticated. Under the manufacturing configuration: 18 SATISFIED control receipts, 9 honest coverage denials, 9 target-divergence MATCH findings, zero silent passes. Continuity and observation rules remained INSUFFICIENT_SOURCE on the baseline case in both conditions — the corpus deliberately omits that evidence and the gap is reported, never filled.
Capabilities that needed no manufacturing code
Canonical evidence pipeline
Structured ingest, canonical envelopes, dedupe, quarantine and persisted evidence sets — the same path every source uses.
Deterministic U5 evaluation
Versioned evaluation with input hash, evidence-set digest, frozen scope snapshot and a canonical disposition. Same engine in both conditions.
Governed control-test framework
FROZEN TestSpecs, durable approvals, witness runs and persisted receipts — the manufacturing spec is configuration on top of this, not a second engine.
Master Assurance Report
An evidence-only report resolves honestly with no scan bound: exactScanId stays null, unsupplied surfaces stay NOT_ASSESSED, the disposition is carried verbatim.
What manufacturing customization added
- Manufacturing semantic pack and ontology — operation, state and identity vocabulary
- Native source profiles and intake normalization for MES/QMS/business-system records
- Five contract rules plus manufacturing detectors (AR-87–96)
- Upstream producer registration with keyed per-record attestation
- Independent commit/readback witness bound to the evaluated transaction
- Manufacturing TestSpec R01 and the evaluation-bound control-maturity facet
What the next manufacturer supplies
- Registered AI systems and deployment environments
- A transport binder per evidence feed — authority role and coverage semantics
- Upstream producer registrations with keyed attestation per native source (MES, QMS, ERP)
- Native operation names, state codes, site/resource/work-order/lot/revision identity mapped into the semantic vocabulary
- Governed TestSpec registration and durable approvals
- Commit and readback evidence on genuinely distinct authenticated emitters
All of the above is configuration and evidence — no new assurance engine, parallel store or second receipt format.
What is not supported today
- OPC UA, MTConnect, Sparkplug B, B2MML and other industrial transports — no adapter or producer registration exists today
- Vendor-native schemas that cannot be expressed through the structured intake contract need a new adapter (engineering, not configuration)
- Continuity and observation coverage where the source does not emit it — gaps are reported, never filled
Static code reachability, repository scanning and agent-discovery surfaces were not exercised by this study — the corpus enters through structured evidence intake. Those generic capabilities exist, but we report what we measured rather than extrapolate.
See what HAIEC can prove for your systems
Start with the generic engine — the same deterministic evaluation this study measured — then customize only where your industry demands it.