Jev failure origin review

Locate the failure shown by a test run

Provide run evidence and the stated expectation. Separate an invoked application's failure from a fixture problem, failed dependency or inconsistent outcomes without an established cause.

Which failure location does this run support?Live API
Try an example

Original synthetic input. Edit the task or evidence to make a new API request.

Source: Workflow inspiration on X

Source and example scope

The inputs, rubric and preview are original. This window demonstrates the stated decision only; it does not reproduce the source application or its measurements.

Read runEvidence.

Decision rules and labels
App failureHarness failureDependency failureVariable evidenceNeed run context
A real response from the Jev API

Edit any input to explore a different decision. No request is sent until you run it.

Your next decision

Not run yet

Choose an example or write your own input, then run it to see the ranked choices.

Results are not stored by this site. Your submitted text is processed by the API provider.

The decision

How does Jev failure origin review use the supplied evidence?

Jev failure origin review locates the failure supported by a supplied run record. Jev separates an observed application failure, a harness failure before invocation, an explicit dependency failure, variable outcomes and missing context. The output proposes which evidence an operator should review next.

A red test result alone does not identify the failed component. The original fixtures describe whether the runner reached the application, what the observed output was and whether an external prerequisite was available. The model reads those supplied facts; it does not execute tests or investigate a live repository.

An API-collected coding-agent post proposed separating bugs, flaky runs and environment problems. It inspired this more conservative evidence-based classification. We do not reproduce its video, claimed speed, spending limits or automatic merge rule. Inconsistent outcomes remain variable evidence rather than a claimed diagnosis of flakiness.

Start with the missing CSV field and the malformed fixture. Both are failures, but only one supplies an observed app result. Then compare the unavailable dependency and the unchanged run pair. All records are fabricated, and each label maps to a pending local review step rather than a retry, patch or merge.

Original Jev failure origin review workflow diagram showing supplied evidence, a typed decision and the local preview.
Original workflow illustration. The output is a local preview, not an executed action or source screenshot.
How it works

Try the example in four steps

Use the sample to understand the rule, then test your own input for test failure location.

  1. Illustration of adding the text or evidence for a decision

    Add the information to check

    Provide run evidence and the stated expectation.

  2. Illustration of defining a question and its allowed choices

    Confirm the question and labels

    Open the decision rules and keep the intended distinction between App failure, Harness failure, Dependency failure, Variable evidence, Need run context. Edit the choices only when testing your own rule.

  3. Illustration of sending one input and comparing the returned choices

    Run the decision

    Use a sample first, then try your own input. Compare the selected label with the other returned probabilities.

  4. Illustration of reviewing a result and an uncertain item before using it

    Review the next step

    Review Need run context before using a result. The label is a decision about the supplied information, not an executed action.

Our API test

What the six original inputs returned.

What the six original inputs returned.
Original test inputFixture expectationObserved labelLeading probability
The invoked app violates the fixture expectationApp failureApp failure100%
The fixture cannot load before invocationHarness failureHarness failure100%
A required remote service failsDependency failureDependency failure100%
Unchanged inputs return different outcomesVariable evidenceVariable evidence99%
FAIL without execution evidenceNeed run contextNeed run context100%
The app returns a different field contractApp failureApp failure100%
  • 6 of 6 original fixture expectations matched. Contrary outputs are preserved in the table and downloads; these checks are not held-out accuracy or production suitability.
  • Measured adapter round trips: 498-1293 ms. Estimated input cost: $0.00002453-$0.00002583 per request. These observations are not a latency guarantee or invoice.

6 actual Jev API calls on October 4, 2026. Original synthetic inputs check this rubric and handoff; they do not establish held-out accuracy. Probabilities are model outputs.Verified

Understand the results

The invoked app violates the fixture expectation

The first record says setup succeeded and the application was invoked.

Read the example notes

It then supplies an explicit expectation and an incompatible observed header. App failure therefore proposes reviewing the application behavior. It does not identify the faulty line or prove why currency was omitted. The wording separates a supported failure location from an unsupported root-cause story, and the packet leaves any real code change under separate review.

Using the result in your application

The malformed JSON prevents the runner from loading its own fixture.

Read the example notes

The record explicitly states that the application was never invoked. Harness failure proposes checking test setup before assigning a defect to application output that does not exist. This does not mean a real application is correct. It means this fabricated run cannot supply evidence about its behavior because the test did not reach that stage.

Source and example scope

The remote service returns an explicit unavailable response before the app assertion.

Read the example notes

Dependency failure proposes reviewing that prerequisite and its authoritative status. The model does not contact the service or establish whether retrying would be safe. A real integration must use its own retry policy, request identity and side-effect rules; a semantic label alone cannot authorize another paid or state-changing call.

The unchanged run pair has different outcomes without an established cause

The unchanged run pair has different outcomes without an established cause.

Read the example notes

Variable evidence preserves that observation without calling the test flaky or blaming a network. The local step asks for review of the run records. Differences in hidden state could matter, but none are supplied here. Additional evidence requires a new request rather than silently relabeling this saved pair as a verified diagnosis.

A bare FAIL gives no reliable failure location

A bare FAIL gives no reliable failure location.

Read the example notes

Need run context requests the missing fixture, invocation or observed output. The model must not infer a familiar CSV problem from the neighboring examples. The original incomplete input is retained in the exact request download. A provider error during the Jev call would be recorded as a separate technical failure, not substituted for this intentionally incomplete run record.

The app returns a different field contract

The final record shows a contract mismatch after successful setup and invocation.

Read the example notes

App failure proposes reviewing the returned field contract rather than the test loader. It does not generate a patch or claim that renaming one property is sufficient. The saved input and full model distribution are inspectable together, and the offline consumer prints only the fixed pending review packet for this declared evidence.

Source and example scope

Fixture expectations are authored before the calls. Any contrary model choice remains visible beside its expectation and complete distribution; no repeated judgment is used to obtain a preferred label.

Using the result in your application

Download the exact sample requests and saved observations below.

Read the example notes

Expected labels are original fixture checks and stay outside the request sent to Jev. The observation file preserves the complete probabilities and the hash of each exact request. Editing a task, rubric or candidate requires a fresh response; a matching sample name does not make an older answer current.

Using the result in your application

The JavaScript runner makes one independent SDK request for a chosen sample ID.

Read the example notes

It reads TYPESAFE_API_KEY privately from the environment and has no automatic retry. The Python consumer operates offline: it checks the rubric, labels, unique sample identities, request hashes and valid probability distributions before selecting a fixed local preview. A tied leading result remains pending for review.

Source and example scope

The page reports measured adapter round trips and estimated input cost for the recorded calls.

Read the example notes

Those values describe these requests, not a provider latency guarantee or a billing statement. Six deliberately authored fixtures can check the stated behavior, but they do not establish accuracy on an unseen dataset. Keep disagreements and missing-input examples when testing an integration.

Runnable offline example

Replay one request and inspect the saved preview.

View the code example
Replay commands
npm install @typesafe-ai/sdk@0.6.0
# Set TYPESAFE_API_KEY privately in your shell.
node run-choice-sample.mjs samples.json app-output
python preview_choice_workflow_v2.py samples.json saved-api-results.json
Runnable offline example

The first request and its recorded local output.

View the code example
Exact first sample and local preview
{
  "input": {
    "task": "Locate the failure supported by this fabricated run record, without inferring an unseen root cause.",
    "runEvidence": "Original fabricated run: fixture loaded successfully; runner invoked buildInvoiceCsv with the stated input. Expected header contains currency. Observed app output omits currency. The fixture recorded an assertion failure after invocation."
  },
  "localOutput": {
    "sampleId": "app-output",
    "leaders": [
      "App failure"
    ],
    "probabilities": {
      "App failure": 1,
      "Harness failure": 0,
      "Dependency failure": 0,
      "Variable evidence": 0,
      "Need run context": 0
    },
    "action": "review_application_failure_evidence",
    "payload": {
      "failureLabel": "App failure",
      "retryStarted": false,
      "patchApplied": false
    },
    "pending": true
  }
}
Scope of this demo

Where does this decision fit?

Useful with explicit supplied evidence

  • Locate the failure supported by a supplied run record: observed app behavior, harness setup, external prerequisite, variable outcomes or missing evidence. Propose a review step without inferring root cause or executing retries.
  • Inspecting the original task and a fixed output before integrating external tools.
  • Comparing an explicit expected label with the complete recorded distribution.

Needs separate implementation or evidence

  • Inferring facts absent from the supplied input.
  • Treating a semantic selection as authorization to act.
  • Executing an external action or generalizing six fixtures into an accuracy claim.
Example FAQ

Questions about this original decision workflow.

Keep relevance, compliance and execution separate.

Does Variable evidence diagnose a flaky test?

No. It preserves different outcomes under the stated unchanged conditions without claiming a cause. A diagnosis needs additional run, environment and execution evidence.

Does Harness failure prove the app is correct?

No. The run did not reach an app check, so it supplies no observed application behavior. It identifies the supplied setup failure only.

Can Dependency failure authorize a retry?

No. The packet proposes review and executes nothing. Retry safety, idempotency and budget remain application rules.

Are these records logs from a real repository?

No. They are original fabricated run summaries designed to make the evidence boundary inspectable. Only the Jev responses and independent SDK and edited-input calls are real API observations.

Try a task from your own workflow

Change the supplied task or evidence here, then inspect the live response. Playground supports another explicit decision.