SpoolisDocs

Spoolis docs

Judge adapter mapping

Map evaluator, CI, test, and review outputs into the generic Spoolis judge adapter contract.

Keep the system that already judges your work. A thin adapter normalizes its output, while Spoolis binds the result to the agreement and evidence and emits the signed Outcome.

The generic contract

binaryJudge accepts one boolean pass result or a per-unit array. scoredJudge accepts quality and consensus values from 0 through 100, either once or per unit. The agreement declares both scored thresholds and a schema ID. Unit IDs must be nonempty and unique.

Minimal adapter outputs
binary aggregate: { "pass": true }
scored aggregate: { "quality": 92, "consensus": 85 }
binary units: { "units": [{ "id": "row-1", "pass": true }] }
scored units: { "units": [{ "id": "row-1", "quality": 92, "consensus": 85 }] }

Spoolis derives pass, fail, or uncertain from these inputs. Do not include a verdict beside the adapter input. The normalized external result also binds the evaluator identity, condition reference, agreement hash, evidence root, observation time, and proof.

Map existing judges

  • Braintrust and LangSmith: adapt a final boolean evaluation to { pass }, or adapt declared 0–100 quality and consensus measurements to { quality, consensus }.
  • Custom CI and deterministic test suites: reduce the declared suite result to { pass }, or preserve test-case identity in the binary per-unit array.
  • Human review: map the reviewer's explicit decision to { pass }. The external judge declaration records the evaluator identity and proof policy.
  • Marketplace evaluators: use the same binary or scored shapes. Marketplace ownership does not change the Spoolis contract.

Proof requirements

proof_requirement is declared, hashed_ref, or signed. Declared proof adds { kind: "declared" }. Hashed reference proof requires a reference and its SHA-256 digest. Signed proof requires a signature reference and the judge declaration must include public key material. Missing or invalid required proof becomes uncertain or fail according to the declared policy. It never becomes acceptance.

Adapter boundary

The adapter translates a provider result. It does not train evaluators, run experiment management, or make a vendor schema part of the Outcome. Spoolis proves which declared judge produced the normalized result and what it was bound to. It does not prove that the judge was substantively correct.

Judge adapter mapping · Spoolis