Jev vs an LLM

The same decision, written both ways

Route a support ticket to one of four queues. Here is what each side actually costs you in code.

const res = await llm.chat({
  messages: [{ role: "user", content:
    `Route this ticket. Reply with JSON: {"queue": "Billing"|"Technical"|"Account"|"Sales"}
     Ticket: ${ticket}` }],
  response_format: { type: "json_object" }
});

let queue;
try {
  queue = JSON.parse(res.content).queue;        // may not be JSON
} catch {
  return retry();                               // ...so there is a retry
}
if (!QUEUES.includes(queue)) return retry();    // may be a queue you never had
// and no idea how close the call was

const { answers } = await jev.systemOne({
  state: ticket,
  model: "jev-latest",
  questions: { queue: { type: "choice", instructions:
    "Which team should own this ticket?", criteria: QUEUES } }
});

const { choice, probabilities, confidence } = answers.queue;
// choice is one of QUEUES. It cannot be anything else.
if (confidence < 0.7) return humanReview(ticket, probabilities);
route(choice);

The retry loop is the tell. It exists in the first version because the answer arrives as text that might not parse, or might parse into a queue that was never on the list. In the second it has nowhere to live: the answer is one of QUEUES by construction. What replaces it is a different branch entirely — one that fires when the model is unsure rather than when it is malformed.

What the second version can see

That ticket, run through Jev for real. The winner is unsurprising; the runner-up is the part a bare label would have thrown away.

Technical wins at 87%, but Account holds 13% — and that is the right disagreement to have, because an SSO failure really does sit on the line between the two. A bare label would have hidden it. If your routing has a cost for getting this wrong, 13% is the number to threshold on.

Side by side

What you send

Jev

One state plus a map of typed questions.

LLM

A prompt, usually carrying the schema and the rules as prose.

What comes back

Jev

One typed answer per question, with a probability for every option.

LLM

Text. JSON if you asked nicely and the mode is supported.

Your next line of code

Jev

A switch on a value that can only be one of yours.

LLM

A parse, then a validation, then a branch for when either failed.

How it fails

Jev

It can be wrong about your text. It cannot return a shape you did not define.

LLM

It can also be wrong about the shape — a missing field, a label you never listed, prose wrapped around the JSON.

Asking ten things

Jev

Ten questions in one request, evaluated against the state in parallel.

LLM

One crowded prompt, or ten calls.

Uncertainty

Jev

A distribution and a confidence value, on every Choice and Score.

LLM

Whatever the text claims about its own certainty.

The numbers

TypeSafe publishes these; they are the vendor's own figures, not measurements of ours.

What we can measure: across 7 runs made through this site, the median round trip is 387 ms — and most of those requests carried nine questions, not one.

When the LLM is the right tool

Most of the time, if what you need is language. Jev has no opinion it can explain and nothing to say.

• You need prose — a reply, a summary, a commit message, an explanation of the decision rather than the decision.
• You need code written, or a plan, or anything open-ended where the answer set is not known before the call.
• The set of possible answers is genuinely unbounded. Extracting an arbitrary name is not a Choice over 255 options.
• You need a chain of reasoning you can inspect. Jev returns a judgement, not the steps behind it.
• The volume is low and the shape is loose. A retry loop you run twice a day is not worth designing away.

The two compose better than they compete. TypeSafe's own framing is System One for the fast structured judgement and an ordinary model as System Two behind it — screen the input with a Noul, route with a Choice, and hand what survives to the model that can actually write.

Keep reading