161 lines
14 KiB
Markdown
161 lines
14 KiB
Markdown
# Conformance suite for the AIP-23 proof kinds
|
|
|
|
Every component beside this one ships its own verifier and its own way of saying no: the AIP-23 reference verifier prints four
|
|
levels, the payment modules return `reasons`, the market judge `motive`, the dataset module `ok`. This folder adds the one place that
|
|
says, in the same form for all of them, **this artifact passes or does not, at level N, for reason M**, and a set of vectors that any
|
|
implementation, in any language, can be run against.
|
|
|
|
It adds no verification rule. A verdict here is derived from the verdict of the published verifier of that artifact; the rule ids
|
|
are the ones those verifiers name, read from their code.
|
|
|
|
## The one verdict form
|
|
|
|
```json
|
|
{ "v": 1, "kind": "aere-conformance-verdict", "artifact": "payment-receipt", "verdict": "INVALID",
|
|
"levels": [
|
|
{ "level": "integrity", "state": "FAILED", "rules": ["receipt.chain"], "reason": "the transaction on chain is not this payment ..." },
|
|
{ "level": "authenticity", "state": "NOT_MEASURED", "rules": [], "cause": "absent", "reason": "the envelope carries no signature" },
|
|
{ "level": "anchoring", "state": "NOT_MEASURED", "rules": [], "cause": "absent", "reason": "no notarization claimed and no --rpc given" },
|
|
{ "level": "finality", "state": "NOT_MEASURED", "rules": [], "cause": "absent", "reason": "no notarization claimed and no --rpc given" } ],
|
|
"failed": ["receipt.chain"] }
|
|
```
|
|
|
|
| level | what it says |
|
|
|---|---|
|
|
| `integrity` | the artifact is what it says: its hashes bind its content, and the rules it declares (a policy, deadlines, a challenge window, Merkle paths, a payout) hold when recomputed from the evidence supplied |
|
|
| `authenticity` | the signature the artifact carries verifies (today: an ML-DSA signature on an AIP-23 envelope) |
|
|
| `anchoring` | its `statementHash` is recorded in the notary contract, under a state root that a post-quantum anchor covers |
|
|
| `finality` | that anchor's post-quantum certificate verifies. The finality is the one of the chain the artifact was notarized on |
|
|
|
|
Each level is `PASSED`, `FAILED` or `NOT_MEASURED`, with the reason; a level nobody judged says why (`cause`). The verdict is
|
|
`INVALID` when a level failed; `PARTIAL` when nothing failed but something the artifact **claims** was not measured (a notarization
|
|
claim not checked because the run was offline, a signature scheme the verifier cannot check, a timeline that is not terminal);
|
|
`VALID` otherwise, and only with integrity passed. A claim that was not checked is never `VALID`. `failed` is the sorted list of the
|
|
rule ids that refused; the suite compares it as an exact set.
|
|
|
|
A receipt or a market timeline that carries an AIP-23 envelope gets that envelope judged too, by the reference verifier, so one
|
|
verdict covers the artifact: the receipt recorded on testnet 28001 comes out with integrity, anchoring and finality `PASSED` in one
|
|
verdict (`vectori/payment-receipt/company-p1-28001.valid.json`).
|
|
|
|
## What is enrolled
|
|
|
|
`REGULI.json` is derived from the code (`node deriva-reguli.mjs --verifica`): 35 rules of 5 verifiers, 8 artifact kinds.
|
|
|
|
| artifact kind | judged by | rules |
|
|
|---|---|---|
|
|
| `envelope` (every AIP-23 kind) | the AIP-23 reference verifier, run as the program it is (`AERE_VERIFY_PROOF=<verify-proof.mjs>`, published with aere-node) | `envelope.form`, `.integrity`, `.signature`, `.finality` |
|
|
| `payment-risk` (a payment presented as allowed under a declared policy) | `payments/risc.mjs` | `risk.form`, `.asset`, `.counterparty`, `.rate`, `.limit`, `.window` |
|
|
| `payment-receipt` | `payments/chitanta.mjs` | `receipt.binding`, `.amount`, `.policy`, `.risk`, `.chain` |
|
|
| `market-timeline` | `market-settlement/piata.mjs` | `market.form`, `.complete`, `.posting`, `.deadline`, `.window`, `.arbiter`, `.payout`, `.result`, `.replay`, `.proof` |
|
|
| `dataset-record`, `-answer`, `-consistency`, `-sample` | `verifiable-data/baza.mjs` | `dataset.form`, `.commitment`, `.field`, `.inclusion`, `.index`, `.range`, `.order`, `.completeness`, `.consistency`, `.sample` |
|
|
|
|
A module verifier exports the list of rules it can name (`REGULI`; its helper throws on any other) and marks every check of a rule;
|
|
the derivation requires declared == marked == mapped here. A rule added to a verifier and not mapped is a refusal of the derivation;
|
|
a mapped rule without a vector is a refusal of the probe. For the reference verifier, a published program this suite does not edit,
|
|
the rules are its level names, and each of the six places where its `finality` level can refuse is classified as `anchoring` or
|
|
`finality` (a seventh place would be a refusal of the derivation).
|
|
|
|
**Not enrolled**, measured in the same file: `ai-trust` (20 refusal sites; its refusals already carry a level and a code, enrolling it
|
|
is a vector per code), `agents` ledger (17), `identity` (15), `verify-layer` log and tree (36): their refusals are free text, and a
|
|
rule id per refusal comes first. Their own probes and negative controls stay what checks them.
|
|
|
|
## The vectors
|
|
|
|
`vectori/` holds 122 vectors (117 offline, 5 that read testnet 28001), listed with their digests in `vectori/INDEX.json`:
|
|
envelope 54, payment-risk 11, payment-receipt 10, market-timeline 25, dataset 22. None is typed by hand (`node genereaza-vectori.mjs
|
|
--verifica` regenerates them and compares byte for byte):
|
|
|
|
- the 16 AIP-23 kinds come from their producers on fixed inputs (the same envelopes as the published AIP-23 kind vectors);
|
|
- the receipt, the five market timelines and the notarized dataset commitment are the evidence recorded on testnet 28001 on
|
|
2026-10-02 beside those components;
|
|
- `surse-proaspete.json` holds what needed a key, randomness or the chain, made once by `--proaspat`: a tree head signed with an
|
|
ML-DSA-65 key that was generated and discarded in that call, an envelope with random content that nobody notarized, a dataset of
|
|
16 records with random salts, and the transaction of the recorded receipt read from the chain (two read calls).
|
|
|
|
An `INVALID` vector names its `base` and its `mutation`: **one operation** (set, remove or add at one JSON path) applied to the input
|
|
of a vector that is not invalid, and it must be refused for exactly the rules in `expected.failed`, not merely refused. Where the rule
|
|
sits under a hash, one place is not reachable (changing an amount under the instruction hash trips the binding first): six vectors
|
|
are then made at the producer, the commitments above the change recomputed with the published formulas, and every path that moved is
|
|
declared (`mutation.changed`). Three vectors expect two rules: rules that guard the same field cannot be separated (a looser policy
|
|
shown under a receipt also makes the risk verdict not reproduce).
|
|
|
|
## Run it
|
|
|
|
```
|
|
AERE_VERIFY_PROOF=<verify-proof.mjs from aere-node, with npm install done beside it>
|
|
node conformance.mjs --impl "node implementare-aere.mjs" # the published verifiers behind the one verdict form: 117 offline vectors
|
|
node conformance.mjs --impl "node implementare-aere.mjs" --online # and the 5 that read testnet 28001 (nothing is sent)
|
|
node conformance.mjs --impl "python3 implementare-integritate.py" # an independent implementation: 55 vectors, 67 reported NOT COVERED
|
|
node conformance.mjs --impl "<your command>" # yours
|
|
node implementare-aere.mjs <artifact.json> [--online] # one artifact { artifact, input }: its verdict
|
|
```
|
|
|
|
An implementation is a command that answers three calls with JSON on stdout: `--capabilities` (`{ artifacts, rules, online, batch }`:
|
|
what it claims to judge), `<vector.json> [--online]` (the verdict above for the file's `{ artifact, input }`), and, if it says
|
|
`batch`, `--batch <list.json>` (one verdict per line). A vector is scored only if the implementation claims its artifact kind and
|
|
every rule the vector exercises; otherwise it is reported `NOT COVERED`, never passed. A scored vector passes when the verdict, the
|
|
exact set of failed rules and the state of each of the four levels are the expected ones. No output is `BROKEN`: a crash is not a
|
|
refusal. Without `--online` an implementation must not open a connection.
|
|
|
|
`implementare-integritate.py` is the second implementation: Python, standard library only, written from the rules and not from the
|
|
JavaScript. It claims 6 of the 35 rules (the form and the statement hash of an envelope; the form, the commitment, the salted field
|
|
commitments and the RFC 9162 inclusion of a dataset record) and passes all 55 vectors it claims. It cannot check an ML-DSA signature
|
|
or read a chain, says so in its capabilities, and the runner reports the rest as not covered.
|
|
|
|
## How the suite itself is checked
|
|
|
|
Measured on 2026-10-03 (Node.js 24.14.1, Python 3.14.4, testnet 28001 read):
|
|
|
|
- `node proba-conformance.mjs --online`: every row green, 10 in this copy (an eleventh, the comparison of the 16 kind envelopes with
|
|
the published AIP-23 kind vectors, runs where their generator is, in the development repository: 11/11 there, and here it is
|
|
printed as information). The rules and the vectors are what the code derives now; every rule has a vector that
|
|
expects it; every kind has a valid vector; every invalid vector is its base with one operation (80) or one declared producer change
|
|
(6); the published verifiers pass 122/122; the independent implementation passes all 55 it claims; and 6,847 malformed inputs (every
|
|
path of every vector removed, or replaced by null, a string, a number, a list, an object) given to the module verifiers each got a
|
|
verdict, none threw. Without `--online` the five chain vectors are reported unmeasured and the exit code is 2.
|
|
- `node proba-conformance.mjs --control-negativ --online`: 35 of 35 rules. Each rule is planted out of its verifier, one at a time, in
|
|
a copy (every check that carries its marker made true at run time); the suite is run against the planted verifier; the vectors that
|
|
expect the rule, **and only they**, must fail, and none may break. Then the instruments: an implementation that calls everything
|
|
valid fails every vector that is not plainly valid; the coverage gate goes red when one rule's vector is taken away; a rule added
|
|
to a verifier and not mapped is refused by the derivation; a judge that throws again turns the malformed-input row red.
|
|
- `node proba-conformance.mjs --pe-loc` is a measurement, not a gate: each of the 60 marked checks of the module verifiers planted
|
|
out alone. For 46 of them a vector then fails. The other 14 are checks whose vector trips a second check of the same rule at the
|
|
same time, so removing one alone changes nothing (the three checks of `market.replay`, the run and neighbour checks of
|
|
`dataset.completeness`), or checks no vector reaches (a dispute timed out before the arbitration timeout: no recorded timeline has
|
|
one). The control above shows that each **rule** is load-bearing, not each of its checks.
|
|
|
|
On its first day the suite found three defects in the verifiers it enrols, all fixed the same day and each now a vector or a probe
|
|
row: a malformed payment made the risk module throw instead of refusing by name; malformed dataset answers made the dataset verifier
|
|
throw; and the market judge accepted a timeline from which the submission event had been withheld, which skipped the challenge-window
|
|
rule (the judge now requires every event the job state implies: `market.complete`).
|
|
|
|
## What it is not, and what is not measured
|
|
|
|
- **Not a certification, not an audit, not a mark.** Passing says that an implementation agrees with the published verifiers on these
|
|
122 inputs. It does not say either of them is correct, and no third party has reviewed the verifiers, the vectors or this suite.
|
|
- **The vectors and the verifiers have the same authors.** The expected verdicts were written by the people who wrote the verifiers.
|
|
The second implementation is independent in code and language, not in authorship, and it covers 6 rules of 35.
|
|
- **Finality is the chain's.** The level says that the post-quantum certificate of the covering anchor verified on the chain where the
|
|
artifact was notarized. Every notarized vector here is on chain 28001, a testnet; nothing in this suite ran on mainnet.
|
|
- **Five of the six finality refusals have no vector.** The reference verifier can refuse at its finality level in six places; the
|
|
vectors reach one (a notarization claimed that the chain does not hold). The others need an endpoint that lies or an anchor that
|
|
does not verify, and are not exercised here.
|
|
- **The chain vectors depend on the testnet's state.** The notary contract is open: anyone can notarize the fresh envelope's hash on
|
|
28001, and `envelope/fresh-28001.*` then change verdict. The runner would show it; `--proaspat` makes a new one.
|
|
- **Integrity is over the evidence supplied.** A market timeline and the transaction facts of a receipt are the caller's reading of the
|
|
chain; this suite does not certify that reading under an anchor. The market judge now refuses a timeline that lacks an event the
|
|
job state implies; it does not prove that the job state shown is the chain's.
|
|
- **A notarization time is taken from the claim.** `market.proof` compares the envelope's claimed first-seen time with the challenge
|
|
window; the reference verifier certifies the time it reads from the chain, and nothing here compares the two.
|
|
- **Authenticity is one scheme.** The reference verifier checks ML-DSA signatures on envelopes; a hybrid signature (as on identity
|
|
credentials) is reported unmeasured and the verdict is `PARTIAL`.
|
|
- **What a damaged artifact can still pass.** Of the 6,847 malformed inputs, 1,810 were still accepted, at 81 distinct paths: parts a
|
|
verifier treats as optional (the transaction facts of a receipt, the envelope of a timeline) and fields no rule reads
|
|
(`node proba-conformance.mjs --acceptate` lists them). A policy that cannot be defined makes the risk module throw, by design. The
|
|
reference envelope verifier is run as published: given the JSON value `null` as an envelope it exits with an exception instead of
|
|
a verdict (1 of 88 malformed envelopes tried on 2026-10-03); the runner scores that `BROKEN`.
|
|
- **Not part of AIP-23.** The rule ids, the four levels and the verdict form are this suite's reading of the published verifiers;
|
|
AIP-23 defines the envelope. Four verifiers published beside these are not enrolled.
|
|
- **The Python implementation** writes numbers the way JavaScript does only for integers and plain decimals, and sorts keys by code
|
|
point (JavaScript's order for keys in the Basic Multilingual Plane).
|