Refund is $90.00 short
Expected
Actual
Evidence limits: This verdict uses the retained business records. It does not establish how the agent executed the request.
Investigation finding
Sandbox diagnosis supported · historical cause unverified
The agent derives the refund amount from the request incorrectly, and the defect depends on the size of the amount: above some magnitude the amount it sends to the payments provider differs from the amount requested.Autonomous investigation complete9 completed experiments · 5 competing explanations considered
Inspect hypotheses and experiments →Original agent message
We've refunded $100.00 to your original payment method for order ord_mvv2n6j6y4nqf8h. It should appear within 5-10 business days.
Needs attention · 1 check
Satisfied checks · 3
Not applicable · 1
When the support agent approves a refund request and resolves the ticket, the payment for the order named in the ticket must carry a succeeded refund for exactly the requested amount, in the requested currency, within two minutes. Refunds on other payments of the same customer do not count.
- Status
- Resolved
- Agent approved
- yes
- Resolved at
- Oct 30, 2026, 14:32 UTC
- Order quoted
- ord_mvv2n6j6y4nqf8h
- Refund requested
- $100.00
- Source
- Support
- Status
- Paid
- Updated
- Oct 30, 2026, 13:32 UTC
- Total
- $100.00
- Source
- Orders
- Payment
- $100.00 Partially refunded
- Refunds
- Succeededre_mvv2n6ma11jg7jlx$10.00
- Refunded
- $10.00 of $100.00
- Source
- Payments
- Shipments for order
- 0
Deadline passed with refund_amount_correct unmet.
Evaluated Oct 30, 2026, 14:35 UTC · deadline Oct 30, 2026, 14:34 UTC · systems Stripe · the snapshot above is the only state Veycera stores for this verdict.
Record identifiers and policy provenance
Ticket tkt_mvv2n6k010qple5v · incident inc_mvv2n75o1bqot461 · execution out_mvv2n6rx149aygte.
Evaluated contract v1. Open current policy
Findings and experiments
Investigation evidence
Diagnosis supportedSupported diagnosis
Evidence ladder
- Mechanism reproduced · The failure was produced in the sandbox under incident-like conditions (2 of 2 runs).
- Diagnosis supported · h_amount_mapping_conditional is supported by discriminating runs; the one-factor intervention x_amount_small removed the failure; alternatives were contradicted.
- Historical attribution · Repeated sandbox failures and a matching business-state signature do not establish the original execution. Incident-linked execution evidence is required.
- Highest rung reached: Diagnosis supported. The sandbox evidence does not establish the original execution.
Reproducible conditions
- Fails with customer agent, refund of 10000 minor units, charge 10000 usd; prior refunds 0, 1 order(s), order id quoted.
- Passes with customer agent, refund of 2000 minor units, charge 10000 usd; prior refunds 0, 1 order(s), order id quoted (only the changed factor differs).
- In 2 of 2 failing agent run(s), the agent resolved the ticket with a completion message.
- Under the incident's conditions the failure appeared in 2 of 2 runs (including the reversal run, which restored it).
- Bisected: a request of $95.00 is refunded correctly, a request of $100.00 is not. The defect switches on between them.
- Requested → issued in agent runs: $20.00 → $20.00, $60.00 → $60.00, $80.00 → $80.00, $90.00 → $90.00, $95.00 → $95.00, $100.00 → $10.00.
Tested range
Suggested fix
- Compute the refund amount once, in the provider's minor units, from the structured requested-amount field, and use the same conversion path for every amount; audit the handling of amounts at or above the observed boundary.
- Quote the amount from the provider's refund response in the confirmation, never from the request.
- Add an outcome check (refund amount equals requested amount) before resolving the ticket.
What remains unknown
- 4 hypotheses were contradicted; 0 remain consistent with every run but not established.
- What the customer agent attempted at the time: Veycera has no logs, traces or tool-call history for it.
- The payments provider's response to whatever refund request was made, and how the agent derived the amount it sent.
- The historical run itself was not observed: Veycera has no logs, traces or execution history of the agent, only the business records and the sandbox runs it executed.
- Even repeated failures, a successful intervention, reversal and a matching business-state signature cannot establish the historical execution. Another mechanism with the same signature remains possible without incident-linked evidence.
- Tested only in a disposable sandbox with synthetic records, covering requested amounts 2000, 6000, 8000, 9000, 9500, 10000 minor units (currency usd). Values outside this range, other currencies, other customer configurations and the production environment itself were not tested; conclusions do not extend to them.
Assumptions behind this finding
- Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge).
- Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption.
- Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history.
- Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors.
- Target version was not supplied; equivalence to the historical target version is unverified.
Competing hypotheses1 supported · 4 contradicted · 0 consistent, not established
Reasoning and technical details
Supported by discriminating runs x_reproduce, x_control, x_direct_incident, x_amount_small, x_reversal; its predicted failure was produced in the sandbox; no run contradicted it and every alternative was separated from it (under the sandbox conditions tested).
Mechanism (observable in the sandbox): Above some requested amount the agent sends the payments provider an amount that differs from the one requested, while smaller requests are refunded exactly; the provider itself accepts the full amount.Implementation guess, unverified: Two code paths derive the provider amount and the one taken for larger requests applies a different unit or scale. Unverified: Veycera does not read the agent's code.For: obs_refund_amount_mismatch, obs_request, obs_cond_refund_amount_correct, obs_ticket_claims_successDistinguished by: Agent refunds of the incident amount versus a small amount, plus a direct API refund of the incident amount: only the large agent run fails.Reasoning and technical details
Contradicted by x_control, x_amount_small (held only in x_reproduce, x_direct_incident, x_reversal), under the sandbox conditions tested.
Mechanism (observable in the sandbox): Every agent refund, whatever its size, is issued for an amount that differs from the request by the same factor.Implementation guess, unverified: A single wrong unit or scale conversion (for example major versus minor units) is applied to every amount. Unverified: Veycera does not read the agent's code.For: obs_refund_amount_mismatch, obs_request, obs_cond_refund_amount_correctDistinguished by: A small-amount agent refund: a uniform defect fails it, a size-dependent one does not.Reasoning and technical details
Contradicted by x_direct_incident (held only in x_reproduce, x_control, x_amount_small, x_reversal), under the sandbox conditions tested.
Mechanism (observable in the sandbox): The payments provider reduces or rejects refunds above a limit; the agent sends the right amount but quotes the request rather than the provider's response.For: obs_refund_amount_mismatch, obs_refunds, obs_ticket_claims_successDistinguished by: A direct API refund of the requested amount against the same charge: a provider rule fails it without the agent involved.Reasoning and technical details
Contradicted by x_reproduce, x_control, x_amount_small, x_reversal (held only in x_direct_incident), under the sandbox conditions tested.
Mechanism (observable in the sandbox): The agent resolves the ticket without performing the operation at all, so no record of the customer changes.For: obs_cond_refund_amount_correct, obs_ticket_claims_success, obs_ticketDistinguished by: Any run in which a record of the customer visibly changed rules out omission; a benign control that still fails supports it.Reasoning and technical details
Contradicted by x_reproduce, x_reversal (held only in x_control), under the sandbox conditions tested.
Mechanism (observable in the sandbox): The write succeeded but the read model lagged behind, so the snapshot was taken before the record reflected it.For: obs_order, obs_refunds, obs_cond_refund_amount_correctDistinguished by: Sandbox runs read after the deadline: lag predicts passes everywhere.Experiments9 completed · isolated sandbox
Every experiment ran in a disposable sandbox with synthetic customers, orders, payments and shipments, against the same black-box agent. Nothing touched production, and each sandbox was disposed of afterwards. Budget: 14 experiments, 60 s, 60,000 tokens / $1.00 of model usage; sandbox writes only, production read-only.
| # | What was varied | Result | What it says about each hypothesis |
|---|---|---|---|
| 1 | Reproduce under incident-like conditions via the customer agent reproducethe agent · refund of $100.00 · 1 orderReplay manifest · constraints preservedPayment $100.00; refundable balance $100.00; currency USD; prior refunds: none. Target: example-support-agent; version unknown; in_process. Provider assumptions: Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge). Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption. Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history. Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors. Replay limits: Target version was not supplied; equivalence to the historical target version is unverified. Experiment wall clock: Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC. Sandbox business clock: Oct 30, 2026, 14:34 UTC. | Refunded $10.00 of $100.00 requested, ticket resolved as doneContract violated; requested 10000, issued 1000 on the named payment; the agent reported completion anyway. |
|
| 2 | Benign control: single order, small refund amount, via the customer agent controlthe agent · refund of $20.00 · 1 orderReplay manifest · constraints preservedPayment $100.00; refundable balance $100.00; currency USD; prior refunds: none. Target: example-support-agent; version unknown; in_process. Provider assumptions: Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge). Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption. Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history. Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors. Replay limits: Target version was not supplied; equivalence to the historical target version is unverified. Experiment wall clock: Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC. Sandbox business clock: Oct 30, 2026, 14:34 UTC. | Conditions metContract verified. |
|
| 3 | Refund the requested amount through the payments API directly, same charge isolate systemdirect API calls · refund of $100.00 · 1 orderReplay manifest · constraints preservedPayment $100.00; refundable balance $100.00; currency USD; prior refunds: none. Target: sandbox-provider-apis; version unknown; in_process. Provider assumptions: Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge). Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption. Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history. Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors. Replay limits: Target version was not supplied; equivalence to the historical target version is unverified. Experiment wall clock: Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC. Sandbox business clock: Oct 30, 2026, 14:34 UTC. | Conditions metContract verified. |
|
| 4 | Same incident, but the customer asks for 2000 instead of 10000 (minor units) counterfactualthe agent · refund of $20.00 · 1 orderReplay manifest · constraints preservedPayment $100.00; refundable balance $100.00; currency USD; prior refunds: none. Target: example-support-agent; version unknown; in_process. Provider assumptions: Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge). Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption. Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history. Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors. Replay limits: Target version was not supplied; equivalence to the historical target version is unverified. Experiment wall clock: Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC. Sandbox business clock: Oct 30, 2026, 14:34 UTC. | Conditions metContract verified. |
|
| 5 | Agent refund of 6000 (minor units), single order boundarythe agent · refund of $60.00 · 1 orderReplay manifest · constraints preservedPayment $100.00; refundable balance $100.00; currency USD; prior refunds: none. Target: example-support-agent; version unknown; in_process. Provider assumptions: Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge). Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption. Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history. Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors. Replay limits: Target version was not supplied; equivalence to the historical target version is unverified. Experiment wall clock: Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC. Sandbox business clock: Oct 30, 2026, 14:34 UTC. | Conditions metContract verified. |
|
| 6 | Agent refund of 8000 (minor units), single order boundarythe agent · refund of $80.00 · 1 orderReplay manifest · constraints preservedPayment $100.00; refundable balance $100.00; currency USD; prior refunds: none. Target: example-support-agent; version unknown; in_process. Provider assumptions: Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge). Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption. Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history. Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors. Replay limits: Target version was not supplied; equivalence to the historical target version is unverified. Experiment wall clock: Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC. Sandbox business clock: Oct 30, 2026, 14:34 UTC. | Conditions metContract verified. |
|
| 7 | Agent refund of 9000 (minor units), single order boundarythe agent · refund of $90.00 · 1 orderReplay manifest · constraints preservedPayment $100.00; refundable balance $100.00; currency USD; prior refunds: none. Target: example-support-agent; version unknown; in_process. Provider assumptions: Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge). Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption. Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history. Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors. Replay limits: Target version was not supplied; equivalence to the historical target version is unverified. Experiment wall clock: Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC. Sandbox business clock: Oct 30, 2026, 14:34 UTC. | Conditions metContract verified. |
|
| 8 | Agent refund of 9500 (minor units), single order boundarythe agent · refund of $95.00 · 1 orderReplay manifest · constraints preservedPayment $100.00; refundable balance $100.00; currency USD; prior refunds: none. Target: example-support-agent; version unknown; in_process. Provider assumptions: Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge). Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption. Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history. Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors. Replay limits: Target version was not supplied; equivalence to the historical target version is unverified. Experiment wall clock: Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC. Sandbox business clock: Oct 30, 2026, 14:34 UTC. | Conditions metContract verified. |
|
| 9 | Reverse the intervention: incident conditions restored reversalthe agent · refund of $100.00 · 1 orderReplay manifest · constraints preservedPayment $100.00; refundable balance $100.00; currency USD; prior refunds: none. Target: example-support-agent; version unknown; in_process. Provider assumptions: Synthetic payments provider rejects a refund exceeding charge amount less succeeded and pending prior refunds (amount_exceeds_charge). Historical provider responses, outages, credentials and attempt-time state are unobserved; synthetic provider behavior is an assumption. Prior refunds are derived only from records timestamped before the request; incomplete or coarse timestamps may omit history. Synthetic fulfillment cancellation cutoff is 30 minutes unless explicitly changed; current availability and access are controlled factors. Replay limits: Target version was not supplied; equivalence to the historical target version is unverified. Experiment wall clock: Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC. Sandbox business clock: Oct 30, 2026, 14:34 UTC. | Refunded $10.00 of $100.00 requested, ticket resolved as doneContract violated; requested 10000, issued 1000 on the named payment; the agent reported completion anyway. |
|
Generated regression testtests/regression/ctr-refund-executed-yc-recording-v1.test.ts
Built from sandbox controls and interventions, not proof of the historical execution. Set an explicit intended VEYCERA_AGENT_URL and an absolute VEYCERA_REGRESSION_SQLITE_PATH for a dedicated disposable harness. Configure the intended agent's VEYCERA_SQLITE_PATH to that same file; a private in-memory database cannot be shared over HTTP. Supply SIM_API_TOKEN if that target requires it. Never use production data. The test checks reproduced-failure and passing-control conditions against the captured contract snapshot.
/**
* Generated by Veycera from an investigation of "Approved refund is issued correctly".
* Evidence level at generation: diagnosis supported.
* Historical attribution is unverified; sandbox reproduction does not prove the original execution.
* Contract snapshot: ctr_refund_executed_yc-recording-v1, version 1.
* Mechanism (sandbox-observed): Above some requested amount the agent sends the payments provider an amount that differs from the one requested, while smaller requests are refunded exactly; the provider itself accepts the full amount.
*
* Target: YOUR agent, reached over HTTP at VEYCERA_AGENT_URL. The test seeds
* the failing conditions in a disposable Veycera sandbox, sends the synthetic
* ticket to that endpoint and asserts the outcome contract against the
* sandbox's business records. It fails until the defect is fixed, and it
* refuses to run against anything but an explicitly configured target.
* Runs inside the Veycera platform package (it uses its sandbox harness);
* it does not import or inspect your agent's code.
* Required: VEYCERA_REGRESSION_SQLITE_PATH is an absolute path to a dedicated,
* disposable SQLite harness shared with the intended agent. Configure that
* agent's VEYCERA_SQLITE_PATH to the same file. A private in-memory database
* cannot be read by an HTTP agent. Never point this harness at production.
* SIM_API_TOKEN is forwarded only to the explicitly configured agent target.
*/
import { describe, expect, it } from "vitest";
import { isAbsolute } from "node:path";
import { createCausalRuntime } from "@/lib/causal/runtime";
import { SandboxLab } from "@/lib/causal/rca/sandbox";
import { experimentFactorsSchema, type ExperimentFactors } from "@/lib/causal/rca/types";
import { outcomeContractSchema } from "@/lib/causal/contracts/schema";
const CONTRACT = outcomeContractSchema.parse({"id":"ctr_refund_executed_yc-recording-v1","tenantId":"yc-recording-v1","name":"Approved refund is issued correctly","description":"When the support agent approves a refund request and resolves the ticket, the payment for the order named in the ticket must carry a succeeded refund for exactly the requested amount, in the requested currency, within two minutes. Refunds on other payments of the same customer do not count.","version":1,"trigger":{"object":"ticket","categories":["refund"],"status":"resolved","requiresAgentApproval":true},"identifiers":{"order":{"source":"ticket.orderRef","required":true}},"conditions":[{"path":"refunds.succeededMinor","op":"eq","ref":"ticket.requestedAmountMinor","id":"refund_amount_correct","description":"Succeeded refunds on the named payment equal the requested amount","applicableWhen":[{"path":"charge.exists","op":"eq","value":true},{"path":"ticket.requestedAmountMinor","op":"neq","value":null}]},{"path":"refunds.succeededMinor","op":"eq","ref":"charge.amountMinor","id":"refund_full_when_unspecified","description":"Without a requested amount, the payment is refunded in full","applicableWhen":[{"path":"charge.exists","op":"eq","value":true},{"path":"ticket.requestedAmountMinor","op":"eq","value":null}]},{"path":"charge.currency","op":"eq","ref":"ticket.requestedCurrency","id":"refund_currency_matches","description":"The payment's currency matches the requested currency","applicableWhen":[{"path":"charge.exists","op":"eq","value":true},{"path":"ticket.requestedCurrency","op":"neq","value":null}]},{"path":"refunds.count","op":"lte","value":1,"id":"refund_single","description":"At most one refund was issued against the payment","applicableWhen":[{"path":"charge.exists","op":"eq","value":true}]},{"path":"siblingRefunds.sinceTicketMinor","op":"eq","value":0,"id":"no_refund_on_other_payments","description":"No other payment of the customer was refunded for this request","applicableWhen":[]}],"instance":{"eventTypes":["ticket.resolved"],"initialRetrySeconds":30,"maxRetrySeconds":600,"maxAgeSeconds":86400},"deadlineSeconds":120,"severity":"high","approved":true,"approvedAt":"2026-10-30T14:32:32.938Z","enabled":true,"draftedBy":"template","provenance":{"sources":[{"kind":"customer_request","ref":"onboarding instruction","excerpt":"Approved refunds must land on the named payment for exactly the requested amount."},{"kind":"policy_document","ref":"Approved refund workflow","excerpt":"issues one refund for the requested amount (minor units, requested currency) against that charge"},{"kind":"api_schema","ref":"Support API (requestedAmountMinor, requestedCurrency), Payments API"}],"assumptions":["A request without a structured amount means a full refund of the payment.","Split refunds are not allowed: at most one refund per payment per request.","Tickets resolved without agent approval (declined or escalated requests) are not monitored."]},"createdAt":"2026-10-30T14:32:32.938Z","updatedAt":"2026-10-30T14:32:32.938Z"});
const FAILING_CONDITIONS = ["refund_amount_correct"];
const AGENT_URL = process.env.VEYCERA_AGENT_URL;
const HARNESS_PATH = process.env.VEYCERA_REGRESSION_SQLITE_PATH;
const EXPECTED_AGENT_VERSION = null;
async function runWorkflow(factors: ExperimentFactors) {
if (!AGENT_URL) throw new Error("VEYCERA_AGENT_URL is not set. This regression test must target your agent's endpoint; it will not fall back to Veycera's example agent.");
if (!HARNESS_PATH || HARNESS_PATH === ":memory:" || !isAbsolute(HARNESS_PATH)) throw new Error("VEYCERA_REGRESSION_SQLITE_PATH must be an absolute path to a dedicated disposable SQLite file shared with the intended agent. Configure the agent's VEYCERA_SQLITE_PATH to the same file; private in-memory and production configuration fallbacks are refused.");
if (EXPECTED_AGENT_VERSION && process.env.VEYCERA_AGENT_VERSION !== EXPECTED_AGENT_VERSION) throw new Error("VEYCERA_AGENT_VERSION must identify the intended tested target version.");
const f = experimentFactorsSchema.parse(factors);
const rt = await createCausalRuntime({ sqlitePath: HARNESS_PATH, agentUrl: AGENT_URL, access: { mode: "deterministic", reason: "regression test" }, env: { SIM_API_TOKEN: process.env.SIM_API_TOKEN, VEYCERA_AGENT_VERSION: process.env.VEYCERA_AGENT_VERSION } as NodeJS.ProcessEnv });
const lab = new SandboxLab(rt.sql, rt.envs, rt.tenantId, rt.agentFor);
try {
const contract = CONTRACT; // immutable evaluated policy, independent of today's stored contract
const env = await lab.create("regression", f);
const fixture = await lab.seed(env.id, f);
await lab.act(env.id, f, fixture);
const { evaluation } = await lab.read(env.id, contract, fixture);
return evaluation;
} finally {
try { await lab.disposeAll(); } finally { await rt.close(); }
}
}
describe("regression: Approved refund is issued correctly", () => {
it("Approved refund is issued correctly: wrong amount computed with customer agent, refund of 10000 minor units, charge 10000 usd; prior refunds 0, 1 order(s), order id quoted", async () => {
const evaluation = await runWorkflow({"workflow":"refund","requestedAmountMinor":10000,"chargeAmountMinor":10000,"currency":"usd","priorRefunds":[],"minutesBeforeDispatch":null,"ordersForCustomer":1,"quotedOrderPosition":"newest","ticketQuotesOrderId":true,"actor":"customer_agent","fulfillmentAccess":"granted","providerAvailability":"up"});
for (const id of FAILING_CONDITIONS) expect(evaluation.conditions.find((c) => c.id === id)?.status, id).toBe("satisfied");
expect(evaluation.status).toBe("verified");
});
it("passing control: customer agent, refund of 2000 minor units, charge 10000 usd; prior refunds 0, 1 order(s), order id quoted (the one-factor intervention that removed the failure)", async () => {
const evaluation = await runWorkflow({"workflow":"refund","requestedAmountMinor":2000,"chargeAmountMinor":10000,"currency":"usd","priorRefunds":[],"minutesBeforeDispatch":null,"ordersForCustomer":1,"quotedOrderPosition":"newest","ticketQuotesOrderId":true,"actor":"customer_agent","fulfillmentAccess":"granted","providerAvailability":"up"});
expect(evaluation.status).toBe("verified");
});
});
Handling audit trail · wall-clock UTC3 entries
- Oct 30, 2026, 14:32 UTCopenedveycera · contract condition(s) unmet: refund_amount_correct
- Oct 30, 2026, 14:32 UTCinvestigationveycera · investigation state open → investigating
- Oct 30, 2026, 14:32 UTCinvestigationveycera · investigation state investigating → diagnosed
Investigation runs
1| Run / wall-clock start (UTC) | Execution | Evidence level | Experiments / error |
|---|---|---|---|
| inv_mvv2n7dn1eq1kkl8Oct 30, 2026, 14:32 UTC · selected | Completed | Diagnosis supported | 9 |
Select runs to compare their reports, manifests and results below. Lifecycle audit entries above record human handling, not experiment evidence. Legacy runs keep their original records; absent inputs are labelled unrecorded.
Selected run provenance
- Agent target
- example-support-agent · in_process
- Target version
- Not recorded; endpoint identity does not establish historical code identity
- Contract
- Captured immutable v1 (ctr_refund_executed_yc-recording-v1)
- Business-state inputs
- Snapshot Oct 30, 2026, 14:35 UTC; original verdict Oct 30, 2026, 14:35 UTC
- Wall-clock execution
- Oct 30, 2026, 14:32 UTC → Oct 30, 2026, 14:32 UTC; 4.0 s
- Budget / usage
- 9/14 experiments; 4040/60000 ms; 0/60000 model tokens; $0.0000/$1.00
- Environment assumptions
- Disposable synthetic sandbox; business-state constraints are recorded per experiment. Agent target version is unknown unless explicitly supplied; an endpoint alone does not establish historical code identity. Provider rules are simulated, not evidence of a historical provider response.
Evidence and investigation log13 observations · 13 wall-clock log lines (UTC)
- obs_cond_refund_amount_correctpaymentsSucceeded refunds on the named payment equal the requested amount: violated (observed 1000, expected refunds.succeededMinor eq ticket.requestedAmountMinor).
- obs_cond_refund_full_when_unspecifiedpaymentsWithout a requested amount, the payment is refunded in full: not_applicable (observed 1000, expected refunds.succeededMinor eq charge.amountMinor).
- obs_cond_refund_currency_matchespaymentsThe payment's currency matches the requested currency: satisfied (observed "usd", expected charge.currency eq ticket.requestedCurrency).
- obs_cond_refund_singlepaymentsAt most one refund was issued against the payment: satisfied (observed 1, expected refunds.count lte 1).
- obs_cond_no_refund_on_other_paymentspaymentsNo other payment of the customer was refunded for this request: satisfied (observed 0, expected siblingRefunds.sinceTicketMinor eq 0).
- obs_ticketsupportTicket tkt_mvv2n6k010qple5v (refund) was approved and resolved at 2026-10-30T14:32:34.919Z; order reference quoted: ord_mvv2n6j6y4nqf8h.
- obs_ticket_claims_successsupportThe resolution message claims completion: "We've refunded $100.00 to your original payment method for order ord_mvv2n6j6y4nqf8h. It should appear within 5-10 business days.".
- obs_no_shipmentfulfillmentNo shipment exists for order ord_mvv2n6j6y4nqf8h.
- obs_ordercommerceOrder ord_mvv2n6j6y4nqf8h is paid; last updated 2026-10-30T13:32:34.747Z.
- obs_refundspaymentsCharge ch_mvv2n6jez96zt9d (10000 usd) has 1 refund(s) totalling 1000 succeeded.
- obs_refundable_balancepaymentsObserved charge amount 10000 usd; 0 refund(s) precede the request, leaving 10000 minor units under the sandbox provider's balance rule. Attempt-time balance and historical provider response remain unobserved.
- obs_requestsupportThe ticket carries a structured refund request for 10000 usd (minor units).
- obs_refund_amount_mismatchpaymentsThe named payment was refunded 1000 usd, not the 10000 requested (charge amount 10000; the refund status is succeeded).
- Cannot know: What the customer agent attempted at the time: Veycera has no logs, traces or tool-call history for it.; The payments provider's response to whatever refund request was made, and how the agent derived the amount it sent.
- Oct 30, 2026, 14:32 UTC Collected 13 observations; failed conditions: refund_amount_correct.
- Oct 30, 2026, 14:32 UTC 5 hypotheses accepted, 0 rejected by validation.
- Oct 30, 2026, 14:32 UTC Planned 4 initial experiments.
- Oct 30, 2026, 14:32 UTC x_reproduce: Contract violated; requested 10000, issued 1000 on the named payment; the agent reported completion anyway.
- Oct 30, 2026, 14:32 UTC x_control: Contract verified.
- Oct 30, 2026, 14:32 UTC x_direct_incident: Contract verified.
- Oct 30, 2026, 14:32 UTC x_amount_small: Contract verified.
- Oct 30, 2026, 14:32 UTC x_amount_boundary_6000: Contract verified.
- Oct 30, 2026, 14:32 UTC x_amount_boundary_8000: Contract verified.
- Oct 30, 2026, 14:32 UTC x_amount_boundary_9000: Contract verified.
- Oct 30, 2026, 14:32 UTC x_amount_boundary_9500: Contract verified.
- Oct 30, 2026, 14:32 UTC x_reversal: Contract violated; requested 10000, issued 1000 on the named payment; the agent reported completion anyway.
- Oct 30, 2026, 14:32 UTC Report: diagnosis_supported.
Simulated time · 6 min ahead
Business records and deadlines use scenario time. Ingestion, reviews, and investigation execution use wall time. Timeline events label their clock; both are shown in UTC. Demos advance business time instead of waiting.