AI Decision System

Change one number in a decision record, and the verifier refuses it.

These packages write a record for each AI-assisted go/no-go decision, with the inputs and frozen rules that produced it. Anyone can recompute the decision offline later, without the system that made it.

The record as written

$ experiment-gate-verify run/decision.json
run/decision.json: verified
exit 0

Treatment conversions edited from 5486 to 5600

$ experiment-gate-verify edited.json
edited.json: mismatch: input_sha256, metrics
exit 1

Output of expgate 0.1.0 from PyPI, run on its bundled offer-ranker-v2 example (synthetic data). The verifier recomputes the statistics from the stored input, so a changed number changes the result.

Three decision points

Three small, independent packages. Each needs only Python 3.11+ and the standard library, and each verifies its own records offline.

DecisionPackageVerify a record
Promote an experiment: SHIP, HOLD, ROLLBACK or INVALID from exported A/B aggregates expgate experiment-gate-verify decision.json
Accept a typed model answer: yes/no, choice or score, gated on labeled cases jevdecision jev-decision-verify answer.json
Approve an action: frozen routing policy, separate human reviewer, local ticket action decisioncontrol decision-control replay --record record.json

RELIA, the model-release gate in the same repository, is not published; its release is human-gated.

Install and verify

pip install expgate jevdecision decisioncontrol

git clone https://github.com/BichengWang/ai-decision-system
cd ai-decision-system
experiment-gate --input ai-workflows/experiment-gate/examples/offer-ranker-v2.json --out /tmp/expgate-run
experiment-gate-verify /tmp/expgate-run/decision.json

Features on main that are newer than the latest PyPI release install from source; the quickstart has the commands and each package's changelog says what is unreleased.

Schemas and conformance vectors

Each JSON Schema is served at the path its $id names. The conformance vectors are records that must verify (valid), must be refused after an edit (tampered) or must fail their schema (invalid), so a verifier in another language can be tested against the same files. Digests use RFC 8785 canonical JSON. The record conventions describe the manifest format.

Schemas

Conformance vectors

What a record does not prove