Jev repair feedback readiness

Check whether repair feedback gives a usable target

Compare the recorded failure with evaluator feedback. Distinguish a specific corrective outcome from vague guidance or a wrong target before planning another attempt.

Does this feedback give the failed run a repair target?Live API
Try an example

Original synthetic input. Edit the task or evidence to make a new API request.

Source: Workflow inspiration on X

Source and example scope

The inputs, rubric and preview are original. This window demonstrates the stated decision only; it does not reproduce the source application or its measurements.

Compare failureSummary with evaluatorFeedback.

Decision rules and labels
Actionable feedbackWrong targetVague feedbackNeed run context
A real response from the Jev API

Edit any input to explore a different decision. No request is sent until you run it.

Your next decision

Not run yet

Choose an example or write your own input, then run it to see the ranked choices.

Results are not stored by this site. Your submitted text is processed by the API provider.

The decision

How does Jev repair feedback readiness use the supplied evidence?

Jev repair feedback readiness checks whether an evaluator has given a failed agent run a concrete repair target. Jev separates a specific corrective outcome, feedback about an unrelated task, vague encouragement and a missing failure report. Code proposes a review step and leaves retries unstarted.

The fabricated failure concerns one CSV export behavior: a currency column is missing. The model receives that failure summary and the evaluator's feedback together. A polished request to try again can still be vague; a shorter instruction can be actionable when it identifies the missing behavior and the required correction.

An API-collected post described a decision-oriented index of public agent patterns, including output validation and source checks. It inspired this original evaluator-to-worker handoff. We have not run the indexed projects, verified their entire repositories or reproduced their reported outcomes. This page implements only the feedback-readiness question shown in the window.

Compare the precise CSV correction with the login-button change, then inspect the missing-run input. An edited feedback message makes a fresh API request. The output cannot establish that a proposed fix is correct: an actual repair would still need code review, a bounded execution policy and tests against the unchanged requirement.

Original Jev repair feedback readiness workflow diagram showing supplied evidence, a typed decision and the local preview.
Original workflow illustration. The output is a local preview, not an executed action or source screenshot.
How it works

Try the example in four steps

Use the sample to understand the rule, then test your own input for repair feedback review.

  1. Illustration of adding the text or evidence for a decision

    Add the information to check

    Compare the recorded failure with evaluator feedback.

  2. Illustration of defining a question and its allowed choices

    Confirm the question and labels

    Open the decision rules and keep the intended distinction between Actionable feedback, Wrong target, Vague feedback, Need run context. Edit the choices only when testing your own rule.

  3. Illustration of sending one input and comparing the returned choices

    Run the decision

    Use a sample first, then try your own input. Compare the selected label with the other returned probabilities.

  4. Illustration of reviewing a result and an uncertain item before using it

    Review the next step

    Review Need run context before using a result. The label is a decision about the supplied information, not an executed action.

Our API test

What the six original inputs returned.

What the six original inputs returned.
Original test inputFixture expectationObserved labelLeading probability
A named failure and corrective outcomeActionable feedbackActionable feedback100%
Try again without a correctionVague feedbackVague feedback100%
A cosmetic change for a data failureWrong targetWrong target100%
No failed-run evidence is suppliedNeed run contextNeed run context72%
Polished encouragement without a repairVague feedbackVague feedback99%
A different precise repair descriptionActionable feedbackActionable feedback100%
  • 6 of 6 original fixture expectations matched. Contrary outputs are preserved in the table and downloads; these checks are not held-out accuracy or production suitability.
  • Measured adapter round trips: 477-1186 ms. Estimated input cost: $0.00002499-$0.00002722 per request. These observations are not a latency guarantee or invoice.

6 actual Jev API calls on October 4, 2026. Original synthetic inputs check this rubric and handoff; they do not establish held-out accuracy. Probabilities are model outputs.Verified

Understand the results

A named failure and corrective outcome

The first feedback names the affected export behavior and the corrective outcome: keep amount and include currency in the CSV header and rows.

Read the example notes

Actionable feedback proposes reviewing that specific repair scope. It does not prove the suggested implementation will compile or pass. The fixture exists only as an authored failure description here. No source code is changed and no test execution is falsely reported.

Try again without a correction

The second feedback repeats that a test failed and asks for another attempt, but supplies no concrete correction.

Read the example notes

Vague feedback proposes requesting a specific outcome before spending another worker run. The label does not judge the evaluator's competence or forbid experimentation. It concerns this handoff under the declared rule. A real worker may need additional logs or code context even after the feedback becomes more specific.

The login-button instruction addresses a different behavior from the supplied data-export failure

The login-button instruction addresses a different behavior from the supplied data-export failure.

Read the example notes

Wrong target preserves that mismatch instead of treating every detailed instruction as useful repair feedback. The local packet asks for feedback tied to the failed behavior. It does not say that the cosmetic change is undesirable in general; it simply cannot explain how that unrelated task would address the recorded missing currency column.

No failed-run evidence is supplied

The fourth fixture has a specific-looking instruction but lacks the failed-run summary.

Read the example notes

Need run context asks for the missing evidence before deciding whether the correction targets the right problem. The model cannot borrow the failure from the five neighboring samples. That independence is deliberate: an evaluator can accidentally attach good feedback to the wrong run, and the request must include its own relevant context.

Polished encouragement without a repair

The encouraging fifth feedback mentions the failing currency check but leaves the desired correction unspecified.

Read the example notes

Vague feedback separates tone from actionable content. This original fixture does not show that Jev generally resists style bias; it checks one declared boundary with one saved response. The full distribution and expected label remain available so a developer can test additional feedback styles rather than extrapolating from the six examples.

A different precise repair description

The last feedback describes the correction in different words and still identifies the currency field, record property and unchanged amount.

Read the example notes

Actionable feedback concerns the semantic repair target rather than an exact match to the first sentence. The packet remains pending for review. Changing the failure, feedback or requirement creates a new request identity; an older observation cannot be reused as proof that a revised repair is now sufficient.

Source and example scope

Fixture expectations are authored before the calls. Any contrary model choice remains visible beside its expectation and complete distribution; no repeated judgment is used to obtain a preferred label.

Using the result in your application

Download the exact sample requests and saved observations below.

Read the example notes

Expected labels are original fixture checks and stay outside the request sent to Jev. The observation file preserves the complete probabilities and the hash of each exact request. Editing a task, rubric or candidate requires a fresh response; a matching sample name does not make an older answer current.

Using the result in your application

The JavaScript runner makes one independent SDK request for a chosen sample ID.

Read the example notes

It reads TYPESAFE_API_KEY privately from the environment and has no automatic retry. The Python consumer operates offline: it checks the rubric, labels, unique sample identities, request hashes and valid probability distributions before selecting a fixed local preview. A tied leading result remains pending for review.

Source and example scope

The page reports measured adapter round trips and estimated input cost for the recorded calls.

Read the example notes

Those values describe these requests, not a provider latency guarantee or a billing statement. Six deliberately authored fixtures can check the stated behavior, but they do not establish accuracy on an unseen dataset. Keep disagreements and missing-input examples when testing an integration.

Runnable offline example

Replay one request and inspect the saved preview.

View the code example
Replay commands
npm install @typesafe-ai/sdk@0.6.0
# Set TYPESAFE_API_KEY privately in your shell.
node run-choice-sample.mjs samples.json specific
python preview_choice_workflow_v2.py samples.json saved-api-results.json
Runnable offline example

The first request and its recorded local output.

View the code example
Exact first sample and local preview
{
  "input": {
    "task": "Review whether the evaluator feedback gives this failed run a usable repair target.",
    "failureSummary": "Original fabricated run: the TestCsvCurrency fixture found that buildInvoiceCsv omits the currency column from the header and each row. The required export contains amount and currency. No other failure is recorded.",
    "evaluatorFeedback": "In buildInvoiceCsv, add the currency header after amount and emit each record's declared currency in that column. Preserve the amount field and check the original TestCsvCurrency fixture."
  },
  "localOutput": {
    "sampleId": "specific",
    "leaders": [
      "Actionable feedback"
    ],
    "probabilities": {
      "Actionable feedback": 1,
      "Wrong target": 0,
      "Vague feedback": 0,
      "Need run context": 0
    },
    "action": "review_specific_repair_scope",
    "payload": {
      "feedbackLabel": "Actionable feedback",
      "retryStarted": false
    },
    "pending": true
  }
}
Scope of this demo

Where does this decision fit?

Useful with explicit supplied evidence

  • Compare a failed-run summary with evaluator feedback to determine whether a specific corrective outcome targets that failure. Propose a bounded review step without judging patch correctness or executing a retry.
  • Inspecting the original task and a fixed output before integrating external tools.
  • Comparing an explicit expected label with the complete recorded distribution.

Needs separate implementation or evidence

  • Inferring facts absent from the supplied input.
  • Treating a semantic selection as authorization to act.
  • Executing an external action or generalizing six fixtures into an accuracy claim.
Example FAQ

Questions about this original decision workflow.

Keep relevance, compliance and execution separate.

Does Actionable feedback mean the fix succeeds?

No. It means the feedback gives the supplied failure a concrete corrective outcome. Correctness, compilation and actual test results require separate execution and evidence.

Why reject a detailed login-button change?

Its specificity does not connect it to the supplied currency export failure. The declared question checks whether the feedback targets this failed behavior.

Can this demonstration start another agent run?

No. It prints a pending review packet with retryStarted set to false. No editor, test runner or agent scheduler is connected.

Why does a specific instruction still need a failure summary?

Without the failed run, the model cannot establish what the correction is meant to repair. The other fixtures are not shared hidden context.

Try a task from your own workflow

Change the supplied task or evidence here, then inspect the live response. Playground supports another explicit decision.