Source and example scope
Replay the saved synthetic requests and compare each observed label with its authored expectation. Any disagreement stays visible; the model is not rerun to obtain a preferred answer.
Supply the message and complete change list. Check whether it describes all independent work, including supporting tests, or leaves part of the commit unexplained.
The export message can be accurate while leaving out a new refund endpoint. This task checks coverage of the supplied change inventory rather than truth of an individual claim.
Coverage does not require a filename list. A regression test for the described bug can be clearly implied. An unrelated endpoint changes the scope and should be represented in the description.
The completeness flag matters. When changed files or hunks were omitted, the result cannot establish coverage of the whole commit. Supply the full inventory before interpreting the label.
In the recorded tests, Jev marked the implied regression test and the vague launch-cleanup description as Unmentioned work. The authored expectations were Covered and Ambiguous. Keep these disagreements visible when deciding how much human review this label needs.
These six synthetic inputs and Choice rules are original demonstrations. The API-collected X post links to jev-commit, whose pinned README describes related checks using Noul questions. This page is not that hook, does not execute Git, and does not reproduce its measurements or credential-blocking policy.

Use the sample to understand the rule, then test your own input for commit change coverage.

Supply the message and complete change list.

Open the decision rules and keep the intended distinction between Covered, Unmentioned work, Ambiguous, Need evidence. Edit the choices only when testing your own rule.

Use a sample first, then try your own input. Compare the selected label with the other returned probabilities.

Review Ambiguous, Need evidence before using a result. The label is a decision about the supplied information, not an executed action.
| Original input | Observed label | Leading probability |
|---|---|---|
| Feature plus tests | Covered | 100% |
| Unrelated endpoint | Unmentioned work | 100% |
| Implied supporting test | Unmentioned work | 58.0% |
| Partial inventory | Need evidence | 68.0% |
| Vague scope relationship | Unmentioned work | 51.0% |
| Explicitly broad coverage | Covered | 83.0% |
These are six original synthetic inputs, tested once per request revision. The percentage is the leading option probability, not measured accuracy. Authored expectations and any disagreements remain in the downloadable evidence. 4 of 6 results matched the authored fixture expectations. For Implied supporting test, the expectation was Covered; the observed choice was Unmentioned work. For Vague scope relationship, the expectation was Ambiguous; the observed choice was Unmentioned work.Verified
Replay the saved synthetic requests and compare each observed label with its authored expectation. Any disagreement stays visible; the model is not rerun to obtain a preferred answer.
The single-request SDK runner uses your private TYPESAFE_API_KEY. The offline consumer reads saved observations, checks request hashes and prints a pending output. It does not call the model or change any external system.
npm install @typesafe-ai/sdk@0.6.0
# Set TYPESAFE_API_KEY privately in your shell.
node run-choice-sample.mjs samples.json case-1
python preview_choice_workflow.py samples.json saved-api-results.jsonInput scope and downstream handling determine what the label can tell you.
No. The rubric accepts a change that is clearly implied, such as a regression test for the stated parser fix.
Diff support asks whether one claim is established by code. Coverage starts from all supplied changes and asks whether the description accounts for them.
No. It is the distribution for this request. Six synthetic fixtures check the workflow and preserve disagreements; they are not a production benchmark.
The window shows an error and returns no fresh judgment. The replay runner makes one request without retries. A missing-evidence label is about the input, not a substitute for an API failure.
Edit the supplied evidence above, or open the Playground for a different decision rule.