Binary criteria alignment: six recorded examples
The downloads contain six original inputs, their expected labels and one recorded API observation per input revision. Disagreements stay visible.
Provide a proposition and both outcome definitions. Check whether true and false match the question, are reversed or introduce a different condition.
This check reads the question and both definitions together. A pair of individually plausible descriptions can still be reversed under the keys true and false. Negative questions need the same explicit reading.
Noul can be called without optional criteria. Needcriteria does not reject that API format; it means this particular alignment review lacks the wording it compares. An alignment result does not explain every wrong model answer or prove the request is accurate.
This original wording-review exercise applies guidance in the repository explicitly named by the cited post. The inputs and review labels are ours. The repository teaches Jev integration; it does not implement this page's analyzer or establish its accuracy.

Use the sample to understand the rule, then test your own input for binary criteria alignment.

Provide a proposition and both outcome definitions.

Open the decision rules and keep the intended distinction between Aligned, Reversed, Inconsistent, Needcriteria. Edit the choices only when testing your own rule.

Use a sample first, then try your own input. Compare the selected label with the other returned probabilities.

Review Needcriteria before using a result. The label is a decision about the supplied information, not an executed action.
| Original input | Observed label | Leading probability |
|---|---|---|
| Matching polarity | Aligned | 97.0% |
| Both labels swapped | Reversed | 96.0% |
| Different condition | Inconsistent | 93.0% |
| Missing false definition | Needcriteria | 95.0% |
| A negative proposition | Aligned | 99.0% |
| Both sides affirm | Inconsistent | 99.0% |
These are six original synthetic inputs, tested once per request revision. The percentage is the leading option probability, not measured accuracy. Authored expectations and any disagreements remain in the downloadable evidence. 6 of 6 results matched the authored fixture expectations.Verified
The downloads contain six original inputs, their expected labels and one recorded API observation per input revision. Disagreements stay visible.
The single-request SDK runner uses your private TYPESAFE_API_KEY. The offline consumer reads saved observations, checks request hashes and prints a pending output. It does not call the model or change any external system.
# Offline: inspect the downloaded saved observations. No API key is needed.
python preview_choice_workflow.py samples.json saved-api-results.json
# Optional live request: install the SDK and set TYPESAFE_API_KEY privately.
npm install @typesafe-ai/sdk@0.6.0
node run-choice-sample.mjs samples.json case-1
# The live response prints to the terminal; it does not update saved-api-results.json.The input contract and these limits define what the result means.
No. The criteria can be optional in the API. This wording check needs both definitions, so its contract expects Needcriteria when either is absent. A model response can still disagree; compare the result with the supplied rule.
True should support the proposition as written. If the question asks whether something is absent, a definition of absence is correctly aligned with true.
It describes the distribution over the supplied choices for this request. These six synthetic examples are functional tests with authored expectations, not a production accuracy benchmark.
No. The demo displays a label and distribution. The downloadable consumer produces a pending preview packet. Application code and a separate authorization policy would control any real action.
Edit the example above to see how a changed fact affects the decision, or open the Playground for another task.