LLM as judge vs verification

An LLM judge is often enough

If the stakes are low, the rubric is informal, and occasional variation is acceptable, asking a model for one overall judgment can be the simplest useful evaluator. LLM judges are especially helpful for exploratory ranking, subjective feedback, and tasks whose meaning cannot be reduced to exact field or file checks.

The limitation is not that a model uses judgment. The limitation is that one prompt and one answer do not, by themselves, create a stable agreement, a reproducible set of checks, an evidence trail, or a payment boundary.

Verification orchestrates several kinds of checks

A verification plan starts with criteria accepted before delivery, then assigns the cheapest reliable verifier to each condition. Exact structure can use deterministic code. Source support or meaning can use semantic judgment when an appropriate verifier exists. A person can remain responsible for conditions that should not be automated.

Spoolis uses AI to help compile transaction intent into proposed terms and verification methods. Proposed safeguards remain separate from what the parties actually agreed. The planner modes are deterministic, ai_assisted, and human_required. Receipt execution methods are a separate vocabulary: deterministic and human_confirmed are live, external_tool is reserved and not live, and ai_assisted is planned and not live.

The decision determines the right evaluator

• Use an LLM judge: the decision is low-stakes, one score or label is enough, variation is acceptable, and no payment action depends on the answer.
• Use a verification plan: criteria must remain stable after agreement, exact checks should be deterministic, failures need evidence, or another system must inspect the result and verify the receipt for authenticity.
• Use a hybrid plan: some conditions are exact while others require semantic or human judgment. The orchestration should preserve the result and evidence for each condition rather than hiding them behind one score.

One plan can combine exact checks and human judgment

A buyer requests a structured research brief that must report status: complete and answer three agreed questions. The status field is an exact deterministic check. Whether the brief answers the questions remains a recorded buyer confirmation in the currently enabled verifier set. Each condition keeps its own method, result, and evidence instead of collapsing into one model score.

The following request uses the production API shape to create that proposed Spool. It requires a bearer key whose owner or party label matches the buyer. The parties must still accept the agreement and submit evidence before verification.

const response = await fetch("https://spoolis.com/api/v1/spools", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.SPOOLIS_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    type: "agent_task",
    parties: [
      { role: "buyer", kind: "agent", display_name: "Buyer agent" },
      { role: "provider", kind: "service", display_name: "Research provider" },
    ],
    value: { amount_cents: 2500, currency: "USD" },
    deliverable: {
      type: "report", title: "Research brief",
      description: "A structured research brief", attributes: { format: "json" },
    },
    terms: [{
      text: "Provide the agreed research brief for $25.00",
      provenance: { source: "user_added", origin: "user_added" },
    }],
    conditions: [
      {
        description: "JSON path \"status\" equals \"complete\"",
        required: true, verification_method: "deterministic",
        deterministic_check: {
          checker: "json_path", path: "status",
          operator: "eq", expected: "complete",
        },
      },
      {
        description: "Buyer confirms the brief answers the three agreed questions",
        required: true, verification_method: "buyer_confirm",
      },
    ],
  }),
})

if (!response.ok) throw new Error(await response.text())
console.log(await response.json())

A signed result preserves authenticity, not absolute truth

When signing is configured, verification can produce an Outcome Receipt containing the agreement and plan versions, condition results, evidence digests, overall result, and earned amount. A third party can verify its authenticity and internal consistency with pinned trust material. The Ed25519 signature proves the receipt is genuine and unaltered, but it does not prove every underlying check is reproducible or that the evidence reflects real-world truth. Evidence provenance describes origin and capture, not trust.

Payment integration remains a separate layer. A buyer-controlled policy can require a valid receipt before taking an allowed payment action. That is different from asking an LLM to authorize money directly.

Use the lightest evaluator reliable enough for the condition

Use the lightest evaluator that is reliable enough for the condition. A single LLM judgment can be the right answer for a low-stakes decision. Add stable criteria, deterministic checks, evidence, human review, reproducibility, and signed receipts only when the transaction needs them.

Read when to use Spoolis for the product-fit decision and fulfillment verification for agentic commerce for the complete loop.