How to Verify an AI Audit Trail Yourself

Ask an AI vendor for evidence of what happened in their system and you will usually get one of two things: screenshots of an admin dashboard, or a CSV export of log rows. Both look like evidence. Neither is.

The problem isn’t that vendors are lying. It’s that nothing about a screenshot or a CSV lets you tell whether they are. The rows are whatever the export said they were, produced by the vendor, from the vendor’s database, using the vendor’s tooling. If an entry was edited last week, the CSV looks identical. If ten entries were quietly deleted, the CSV looks identical. Accepting that as an audit trail means accepting the vendor’s word for the one property an audit trail exists to provide — that the record is complete and unaltered. That’s faith-based evidence, and a compliance officer or examiner is entitled to something better: HIPAA’s Audit Controls standard requires mechanisms to record and examine activity in systems containing PHI,^[1] and examining a record whose integrity you can’t establish isn’t examination. We’ve written about what separates application logs from a compliance-grade audit trail; this post is about the last and hardest property on that list — integrity you can check for yourself — and then a walkthrough where you actually check it, against a real export, with one command.

What makes an audit trail verifiable

Three mechanisms, layered, turn “trust us” into “check the math.” Each one is simpler than it sounds.

A hash chain. Every entry in the log is run through a hash function — a piece of math that turns the entry’s exact contents into a short fingerprint, where changing even one character of the entry produces a completely different fingerprint. Each entry also records the fingerprint of the entry before it, which links the whole log into a chain. Edit any past entry and its fingerprint no longer matches; delete one and the link to its neighbor breaks. Anyone holding the export can recompute every fingerprint and confirm the chain is intact — no access to the vendor’s systems required.

Digital signatures. A hash chain proves the export is internally consistent, but a determined party could rebuild a whole fake chain from scratch. Signatures close that door. Each entry is signed with a cryptographic key — HASP uses Ed25519, a widely deployed modern signature standard^[2] — and the matching public key is published where anyone can fetch it. Anyone can check a signature with the public key; only the holder of the private key can create one. A fabricated or altered entry fails the check, because forging the signature would require a key the forger doesn’t have.

External time anchors. Chain and signatures still leave one move open: the keyholder rewriting history and re-signing it, backdated. So the state of the chain is periodically sent to a Time Stamping Authority — an independent third party with no business relationship to the vendor — which countersigns it with a trusted timestamp under RFC 3161, the internet standard for exactly this.^[3] Once an anchor exists, everything in the chain up to that point is frozen: rewriting it would now require forging the third party’s signature too. The timestamp is the piece that makes “this record existed, in this form, by this date” a claim no vendor can quietly take back.

Together the three properties mean tampering isn’t prevented by policy — it’s detectable by arithmetic, by anyone, on any machine. Which brings us to the walkthrough, because HASP publishes everything you need to try it right now.

The walkthrough: verify a real export

You don’t need a HASP account. The sample export at usehasp.com/trust/audit-export-sample.json is a real export produced by the platform’s own audit system — never hand-edited — carrying real signatures and a genuine RFC 3161 timestamp token. The verifier is an open-source, MIT-licensed command-line tool published on npm; it’s a few hundred lines of code you can read before you run.

Download the sample and run the verifier — one command each:

curl -sLo export.json https://usehasp.com/trust/audit-export-sample.json
npx @usehasp/verify export.json

The expected output:

✓ schema valid
✓ chain intact (4 / 4 entries)
✓ published key matches (key_id 01ky8rr9d9fmydvy94ya042gh8)
✓ signatures verified (4 / 4)
✓ time-attested
  — entries 1..4 time-attested (history commitment)
✓ TSA anchor valid
  — https://freetsa.org/tsr

VERIFIED.

Green means every check passed. Any failure prints exactly which entry broke and why — and a failure on a real export is a finding, reproducible on your machine. If you’d rather not trust our tool either, fair enough: the full manual recipe reproduces every check below using only Python, openssl, jq, and curl — standard tools, no HASP code anywhere in the loop. Same math, same answer.

What each of the six checks proves

The one command runs six checks. Here is what each one establishes, in order.

1. Schema. The export is well-formed: every entry carries the fields the later checks depend on — contents, hash, signature — and the verification block declares the chain head, the public key, and the timestamp anchors. This check exists so nothing can pass by omission; an export that leaves out the verifiable parts fails before anything else runs.

2. Chain. The verifier recomputes every entry’s hash from the entry’s actual contents and confirms each entry correctly references the fingerprint of the one before it, from the first entry through the declared chain head. Passing proves no entry in the export was modified after writing, and none was removed or inserted — any of those breaks the chain at the exact point it happened.

3. Published key. The export embeds the public key its signatures verify against — but a forger could embed their own key next to their own forged signatures. So the verifier compares the embedded key against the key HASP publishes independently at usehasp.com/.well-known/audit-keys.json. A match proves the export is bound to the key the vendor has publicly committed to — not a substitute minted for this one file.

4. Signatures. Every entry’s Ed25519 signature is checked against that published key. Passing proves each entry was signed by the genuine key at write time — an altered entry, or an entry fabricated wholesale, fails because producing a valid signature requires the private key.

5. Anchoring coverage. An anchor is only useful if it actually covers the records in front of you. The verifier confirms every entry in the export is bound into the chain state the timestamp anchor commits to — the “history commitment” — so no entry sits outside the attested history. Passing proves the anchor vouches for these specific entries, not merely for some chain state somewhere.

6. TSA token. Finally, the verifier confirms the RFC 3161 timestamp token itself is genuine — cryptographically signed by the independent Time Stamping Authority. Passing proves the anchored history existed, in exactly this form, no later than the timestamp — the record wasn’t reconstructed after the fact and backdated.

What an auditor can conclude — without trusting the vendor

Put the six together and the conclusion is unusually strong for a vendor-produced document: the records in this export were written as claimed, signed with the vendor’s publicly committed key, have not been altered, added to, or trimmed since, and everything up to the anchor existed by an independently attested date. Every step of that sentence was checked on the auditor’s own machine, against published keys and a third party’s signature. The vendor’s honesty was never an input.

Two honest limits are worth naming, because a verifiable trail should be precise about what it proves. First, entries written after the most recent time anchor are signed but not yet countersigned by the third party; the next anchor covers them, and re-running verification against a later export closes the gap. Second, verification proves the integrity of what was recorded — that the log captures every event it should is a completeness question, answered by how the system is built (in HASP, logging happens at the same gateway that processes every request, so a prompt can’t reach a model without producing an entry) and evaluated the way the audit trail post describes. Integrity you verify; completeness you inspect. A trail that offers neither is asking for faith on both.

Try it before you need it

The best time to run this walkthrough is before an auditor asks — partly to see the green checks once, mostly because it recalibrates what you accept as evidence. Once you’ve watched an export verify itself with math, a screenshot of a dashboard stops looking like an audit trail and starts looking like what it is: a picture of one.

The full verification recipe has the sample export, the published keys, the one-command tool, and the complete manual procedure for auditors who want to check every step by hand. And when a vendor tells you their system has an audit trail, you now have the shortest, most clarifying question in the evaluation: can I verify it without trusting you?

Sources

1. Office of the Federal Register. “45 CFR 164.312 — Technical safeguards,” paragraph (b), Audit controls. eCFR.gov.

   ecfr.gov

2. Internet Research Task Force. RFC 8032: Edwards-Curve Digital Signature Algorithm (EdDSA) (2017).

   datatracker.ietf.org

3. Internet Engineering Task Force. RFC 3161: Internet X.509 Public Key Infrastructure Time-Stamp Protocol (TSP) (2001).

   datatracker.ietf.org