ISAF Logger: Evidence Generation and Lineage for AI Systems

An auditor asked a client of mine last year for documentation of their model training pipeline. The client had trained a fine-tuned model six months earlier. The engineer who built it had left. The training script was in a Git repo somewhere, but nobody could find the exact commit, the exact dataset version, or the hyperparameters used. The auditor wanted proof that the model was trained on the data they claimed, with the objectives they stated, on the date they reported.

The client spent three weeks reconstructing the training history from Git logs, Slack messages, and a Notion page. The result was a PDF that the auditor accepted, but everyone involved knew it was reconstruction, not evidence. It was what someone remembered, written down after the fact.

ISAF Logger exists for this situation. It is a Python package that captures lineage metadata during training and inference — automatically, as the code runs, with SHA-256 hash chains that make the evidence tamper-evident. You add 3 lines of code to your training script. When the auditor asks, you hand them the lineage report and the hash chain.

What ISAF Logger is

ISAF stands for Instruction Stack Audit Framework. It is a Python package published on PyPI as isaf-logger (v0.2.0, MIT licensed).

ISAF logs lineage metadata for Layers 6-9 of the instruction stack:

• Layer 6 — Framework: which ML framework, which version, which configuration
• Layer 7 — Data: which dataset, which version, which preprocessing pipeline
• Layer 8 — Objectives: which loss function, which hyperparameters, which training objective
• Layer 9 — Deployment/Inference: which model version, which endpoint, which inference configuration

Each layer is logged as the code executes. Not after the fact. Not from memory. From the actual runtime state.

The 3-line integration

from isaf_logger import ISAFLogger

logger = ISAFLogger(project="my-ai-model")
logger.log_training_run(model="fine-tuned-v2", dataset="training-set-v1.3", epochs=10)

That is the minimum. The logger captures the framework version, Python version, Git commit hash (if available), dataset hash, and hyperparameters automatically. You do not need to pass them manually.

For more detailed logging:

from isaf_logger import ISAFLogger

logger = ISAFLogger(
    project="my-ai-model",
    storage="sqlite",  # or "mlflow" for production, "memory" for testing
)

logger.log_layer("framework", {
    "framework": "pytorch",
    "version": "2.1.0",
    "cuda_version": "12.1",
})

logger.log_layer("data", {
    "dataset": "training-set-v1.3",
    "hash": "sha256:a1b2c3d4...",
    "rows": 45000,
    "preprocessing": "tokenize -> pad -> truncate(512)",
})

logger.log_layer("objectives", {
    "loss": "cross_entropy",
    "learning_rate": 0.0001,
    "batch_size": 32,
    "epochs": 10,
})

logger.log_layer("deployment", {
    "model_version": "fine-tuned-v2",
    "endpoint": "api.example.com/v2/predict",
    "runtime": "onnx",
})

report = logger.generate_report()
# Returns a JSON lineage report with SHA-256 hash chains

What the evidence looks like

The lineage report is a JSON file with four sections:

{
  "session_id": "isaf-2026-08-22-a1b2c3",
  "timestamp": "2026-08-22T14:30:00Z",
  "layers": {
    "framework": { "framework": "pytorch", "version": "2.1.0", "cuda_version": "12.1" },
    "data": { "dataset": "training-set-v1.3", "hash": "sha256:a1b2c3d4...", "rows": 45000 },
    "objectives": { "loss": "cross_entropy", "learning_rate": 0.0001, "batch_size": 32 },
    "deployment": { "model_version": "fine-tuned-v2", "endpoint": "api.example.com/v2/predict" }
  },
  "hash_chain": {
    "framework": "sha256:e5f6g7h8...",
    "data": "sha256:i9j0k1l2...",
    "objectives": "sha256:m3n4o5p6...",
    "deployment": "sha256:q7r8s9t0...",
    "chain_root": "sha256:u1v2w3x4..."
  }
}

Each layer's hash is computed from the layer's content plus the previous layer's hash. This creates a chain: if anyone modifies Layer 7 (data), the hash for Layer 7 changes, which changes the hash for Layer 8, which changes the hash for Layer 9, which changes the chain root. A single modification breaks the entire chain.

The auditor can verify the chain by recomputing the hashes from the layer contents and comparing them to the recorded hashes. If they match, the evidence has not been modified since it was generated.

CLI tools

ISAF Logger includes CLI tools for inspecting and verifying evidence:

# List all logging sessions
isaf list-sessions

# Inspect a specific session
isaf inspect isaf-2026-08-22-a1b2c3

# Verify hash chain integrity
isaf verify isaf-2026-08-22-a1b2c3

# Export from SQLite database
isaf export-from-db --db evidence.db --output report.json

The verify command recomputes the hash chain and compares it to the stored chain. If any layer has been modified after logging, the verification fails.

Storage backends

ISAF Logger supports three storage backends:

• SQLite (local): default. Stores evidence in a local .db file. Good for development and single-machine training.
• MLflow (production): stores evidence alongside MLflow experiment tracking. Good for teams using MLflow for experiment management.
• Memory (testing): stores evidence in memory. Good for unit tests and CI pipelines where you want to verify logging without persisting.

Compliance mapping references

ISAF Logger includes mapping references to common frameworks:

• EU AI Act Article 10/11: data governance and documentation requirements for high-risk AI systems
• NIST AI RMF: Map, Measure, Manage functions
• ISO 42001: AI management system requirements
• Colorado AI Act: impact assessment documentation requirements

These are references, not certifications. The mapping tells you which ISAF layers correspond to which framework requirements. It does not certify that your system complies with the framework. Compliance requires a qualified auditor to review the evidence and make a determination.

What ISAF Logger is not

Being explicit about this, because the naming causes confusion:

Not a security scanner. ISAF logs lineage metadata. It does not scan source code for vulnerabilities, does not check tenant boundaries, does not inspect model outputs. For security scanning, use AI AppSec, MCP Tenant Isolation, and LLMVerify.

Not a compliance certification. ISAF generates evidence that can support compliance documentation. It does not certify compliance with any framework. An auditor reviews the evidence and makes a determination. ISAF provides the raw material, not the judgment.

Not a model behavior validator. ISAF does not validate whether the model behaves correctly, produces safe outputs, or meets performance requirements. It logs what was used to train and deploy the model, not what the model does when it runs.

Not a replacement for human review. ISAF evidence supports audit and review workflows. It does not replace human judgment, formal audit, or regulatory review. The hash chain proves the evidence has not been tampered with. It does not prove the evidence is complete or accurate.

Where it fits

ISAF Logger is the evidence generation layer in the Developer Security family. It is not one of the three security scanners (source, boundary, runtime). It is a supporting tool that generates the evidence auditors ask for when they review AI systems.

The three scanners find vulnerabilities. ISAF Logger documents what was built and how. Both are needed for a complete AI security and compliance program.

| Tool | What it does | What it does not do | |---|---|---| | AI AppSec | Scans source code for AI vulnerabilities | Does not generate lineage evidence | | MCP Tenant Isolation | Scans tenant boundary logic | Does not generate lineage evidence | | LLMVerify | Checks runtime model I/O | Does not generate lineage evidence | | ISAF Logger | Generates lineage evidence | Does not scan for vulnerabilities |

Install

pip install isaf-logger

ISAF Logger requires Python 3.8+. It supports PyTorch, TensorFlow, JAX, and scikit-learn. The package is MIT licensed and the source is at github.com/haiec/isaf-logger.