Confidence-Gated Routing with Jev

The decision and the gate

Keep two stages in code: a bounded answer selects a possible action, then a risk-aware gate decides whether that action may proceed. Low confidence needs a real destination rather than a log entry followed by the original action.

A help article versus a money-moving operation

A mistaken help article is usually recoverable. A mistaken transfer or refund can lose money. These actions should not share a single “safe” threshold. Even a high-confidence financial recommendation still requires verified policy, identity, balance checks, and explicit confirmation where appropriate.

Example policy

def route(answer, action):
    # Example thresholds only. Evaluate on labeled data.
    if answer.confidence < 0.6:
        return "human_review"
    if action == "show_help" and answer.confidence >= 0.85:
        return "show_help"
    if action == "move_money":
        return "verify_policy_then_request_confirmation"
    return "verify"

Keep fallbacks useful

A review record should carry the input reference, actual model, selected label, distribution, and policy revision. If a human must approve an action, bind the approval to exact arguments and expire it when those arguments change.

Test thresholds

Sweep thresholds over representative labeled input. Measure false automatic actions, missed actions, and review volume separately. Re-evaluate after model upgrades and changes to criteria. Do not lower a threshold merely because the review queue is inconvenient.

Failure mode

Confidence measures reported uncertainty, not truth. Confidently wrong answers exist. Sample automatic decisions for audit and track reviewer corrections. Keep deterministic policy checks in the executor even when the model reports certainty.