Step 10 of 10
Verify it offline
Anyone with the bundle can re-derive every claim with no network and no data. The verifier also says plainly what the bundle does not establish. This is the output of the verifier over the sealed bundle of demo-001, run with no network:
$ histor verify evidence-bundle/
ok bundle_version: is this a bundle format this verifier knows how to check? bundle_version 0.3 ok bundle_format: is the bundle's format the one its ledger was written for? bundle_version 0.3, as the ledger's report_generated (seq 39) records ok signing_keys: was every statement checked under a key the ledger recorded or you gave, not one read from public-keys.json alone? control-plane (ledger seq 2), harness (ledger seq 11), scorer (ledger seq 22) ok signatures: is every attestation the one the ledger recorded, signed by the control plane? 2 attestation(s), each the one the ledger recorded for its run, verify under the control plane's key ok statement_types: is every signed statement of a type this verifier knows? 6 statement(s), each of a known type under https://historlabs.eu/ ok hash_chain: is the ledger hash chain intact, with no gaps? 41 entries, chain intact ok run_numbers: was every run started once, and ended in the ledger, with no unrecorded runs? runs [1, 2], contiguous, each started once and ended in the ledger ok plan_versions: is every plan version the ledger records present in the bundle? 2 version(s) — the plan was amended during the participation warn plan_signatures: was the plan signed by each party, through their own identity provider? the plan was signed with keys the sandbox holds, not through the parties' identity providers: the bundle shows that the sandbox signed it, not who agreed to it warn report_signature: was the exit report signed by the regulator, through their own identity provider? report sha256:934fbb66424e… is signed through the development IdP, which issues a token for anyone: the signature identifies nobody ok artifact_digests: do the artifacts in every run match the pins of the plan it cites? 2 run(s) match the plan's pins ok dataset_commitments: was every dataset used committed before the run that used it? 1 committed warn isolation: is isolation evidence present and does it match the plan's policy? isolation at simulated-centre (histor, no Slurm, no Apptainer; model in a loopback-only netns) (run 1, 2) vouched for by the sandbox operator: the operator's signature over the job record and node configuration its courier observed verifies, and every pinned image has a signed conversion to the SIF that ran. This rests on the sandbox operator's word: the party that runs the sandbox, vouching for its own run. That is weaker than a centre's word, and neither is a network policy anyone can hash or a drop log anyone can read. Run 1's keys were released once, to a key (digest sha256:b727f00df0c5…) held by Slurm job 4201, after that job's measured SIF digests matched the conversions, every probe from the model's namespace was blocked and that namespace held only loopback (ledger seq 21). Run 1's plan pins no encrypted weights, so the model's weights were in its image and passed through the sandbox with it. Run 2's keys were released once, to a key (digest sha256:55d42eaa881d…) held by Slurm job 4202, after that job's measured SIF digests matched the conversions, every probe from the model's namespace was blocked and that namespace held only loopback (ledger seq 35). Run 2's plan pins no encrypted weights, so the model's weights were in its image and passed through the sandbox with it. Root on the compute node could read the model while it ran; what covered that is a contract, the centre's confidentiality undertaking (demo-centre-undertaking), not cryptography. ok run_logs: do the run logs in the bundle match the digests their attestations carry? 8 logs match their digests ok harness_statements: did the harness sign what it measured, and does the attestation agree? 2 run(s): the harness signed its measurements and the attestation agrees; 2 scored off the centre, from the driver's signed observations, which the scorer's statement agrees with ok thresholds: does every stated outcome follow from the numbers reported with it? recomputed and consistent ok sample_sizes: were any thresholds passed on samples too small to mean anything? every scored group met the plan's minimum ok deletion: were keys destroyed after exit? keys destroyed after exit: ['data/demo-001/lab-heldout-v1', 'signing/demo-001/harness', 'signing/demo-001/scorer', 'work/demo-001/run-1', 'work/demo-001/run-2'] ok completeness: does the ledger end as a participation that has ended does? exit, keys destroyed, the report and the regulator's signature over it, and the report's bundle digest matches this bundle up to seq 38 warn timestamps: are the timestamps valid, ordered, and external? every timestamp was issued by the sandbox operator's own development authority, not by a third party. The ordering is self-consistent and anchors nothing: an operator able to rebuild this ledger could reissue these timestamps with it. warn tsa_revocation: was the timestamp authority's certificate unrevoked when it stamped? no RFC 3161 timestamp in this ledger, so no authority certificate whose revocation could be checked: development timestamps carry none ok personal_data: does the manifest say what personal data the bundle holds? as the manifest says, the bundle holds personal data: staff identifiers (7, in ledger.jsonl, plan.json, plans/); idp id tokens (1, in ledger.jsonl), and no test-subject data (none of the files the export writes can carry it) ok i18n_catalogues: is the catalogue each translated rendering of the report was made with in the bundle, as the ledger records it? the report was rendered in English only: no catalogue to carry ok report_renderings: is the exit report rendered in every language the plan names, each rendering named by hash in the authentic report? the plan names no languages: one report, in English warn anchors: were the trust anchors given from outside the bundle? the plan digest, the sandbox id, the timestamp authority's root, the identity providers' keys, the network policy digest, the harness image digest, the statement signing keys, the audience the parties' IdP logins were for, the ledger head came from the bundle itself: the checks against them show that the bundle agrees with itself, not that it is the one you signed. An operator who rebuilt it could have replaced them all consistently. Pass them from outside, with --anchors or the --expect-* options. ok bundle_digest: does the bundle match the digest in its own manifest? sha256:32509c274028b5747968f63e2b018123bdc0aa98aafd482f5575e66880235c9c VERIFIED — 20 checks passed, 6 warning(s) Anchors given from outside: none Anchors read from the bundle: plan_digest, sandbox_id, tsa_root, idp_keys, policy_digest, harness_digest, signing_keys, audience, ledger_head What this bundle does NOT establish: - plan_signatures: the plan was signed with keys the sandbox holds, not through the parties' identity providers: the bundle shows that the sandbox signed it, not who agreed to it - report_signature: report sha256:934fbb66424e… is signed through the development IdP, which issues a token for anyone: the signature identifies nobody - isolation: isolation at simulated-centre (histor, no Slurm, no Apptainer; model in a loopback-only netns) (run 1, 2) vouched for by the sandbox operator: the operator's signature over the job record and node configuration its courier observed verifies, and every pinned image has a signed conversion to the SIF that ran. This rests on the sandbox operator's word: the party that runs the sandbox, vouching for its own run. That is weaker than a centre's word, and neither is a network policy anyone can hash or a drop log anyone can read. Run 1's keys were released once, to a key (digest sha256:b727f00df0c5…) held by Slurm job 4201, after that job's measured SIF digests matched the conversions, every probe from the model's namespace was blocked and that namespace held only loopback (ledger seq 21). Run 1's plan pins no encrypted weights, so the model's weights were in its image and passed through the sandbox with it. Run 2's keys were released once, to a key (digest sha256:55d42eaa881d…) held by Slurm job 4202, after that job's measured SIF digests matched the conversions, every probe from the model's namespace was blocked and that namespace held only loopback (ledger seq 35). Run 2's plan pins no encrypted weights, so the model's weights were in its image and passed through the sandbox with it. Root on the compute node could read the model while it ran; what covered that is a contract, the centre's confidentiality undertaking (demo-centre-undertaking), not cryptography. - timestamps: every timestamp was issued by the sandbox operator's own development authority, not by a third party. The ordering is self-consistent and anchors nothing: an operator able to rebuild this ledger could reissue these timestamps with it. - tsa_revocation: no RFC 3161 timestamp in this ledger, so no authority certificate whose revocation could be checked: development timestamps carry none - anchors: the plan digest, the sandbox id, the timestamp authority's root, the identity providers' keys, the network policy digest, the harness image digest, the statement signing keys, the audience the parties' IdP logins were for, the ledger head came from the bundle itself: the checks against them show that the bundle agrees with itself, not that it is the one you signed. An operator who rebuilt it could have replaced them all consistently. Pass them from outside, with --anchors or the --expect-* options.
Download the demo bundle (synthetic data, public keys only) and check it yourself: tar xzf demo-001-evidence-bundle.tar.gz, then histor verify evidence-bundle/, offline. The verifier's public release is coming.
The warnings are the point: a development timestamp authority, development identity providers and a simulated centre's operator signature are named as what they are, not passed off as more. Back to the tour