Dejev

Score — rating against a rubric in Jev

A Score is one of the three question types in TypeSafe’s Jev model. It rates the input against ordered levels you write and returns a probability-weighted value that can land between two levels.

A Score is a rubric. You write the levels in order, lowest first, in words — and the answer is a position on that scale, numbered from 0. Because it is a probability-weighted mean of the levels, it can and often does land between two of them.

Describe the levels rather than numbering them. TypeSafe’s own documentation shows the same input scoring cleanly against three described levels and splitting its probability when given bare numbers: with nothing to match against, the model has nothing to reason from.

A real answer

Sent to jev-1.13.0. The numbers below are what came back.

The state

CSV export silently truncates at 1000 rows since 4.2.0. No error shown. Downgrading fixes it. Reproduced on two accounts. This breaks our month-end close.

The question

How severe is this issue?

70%
medium
2.35 of 4 · Major, a workflow is broken

2.35 sits just past "a workflow is broken" — but read the distribution rather than the number. 77% is on level 2, and yet 13% sits all the way up on level 4, because silent truncation really can be data loss. A mean of 2.35 and a tight peak at 2.35 are different answers, and only `probabilities` tells them apart.

The request and the response, verbatim
Request
{
  "state": "CSV export silently truncates at 1000 rows since 4.2.0. No error shown. Downgrading fixes it. Reproduced on two accounts. This breaks our month-end close.",
  "model": "jev-latest",
  "questions": {
    "severity": {
      "type": "score",
      "instructions": "How severe is this issue?",
      "criteria": [
        "Cosmetic",
        "Minor, has a workaround",
        "Major, a workflow is broken",
        "Critical, core functionality is unusable",
        "Catastrophic, data loss or a security hole"
      ]
    }
  }
}
Response
{
  "model": "jev-1.13.0",
  "answers": {
    "severity": {
      "type": "score",
      "score": 2.35,
      "confidence": 0.7,
      "legend": {
        "0": "Cosmetic",
        "1": "Minor, has a workaround",
        "2": "Major, a workflow is broken",
        "3": "Critical, core functionality is unusable",
        "4": "Catastrophic, data loss or a security hole"
      },
      "probabilities": {
        "0": 0,
        "1": 0,
        "2": 0.77,
        "3": 0.1,
        "4": 0.13
      }
    }
  }
}

What the answer contains

type "score"
Matches the question type.
score number
The probability-weighted mean of the level numbers. For five levels it runs 0 to 4, and it can land between levels.
legend map<string, string>
Each level number mapped back to the description you wrote.
probabilities map<string, number>
Each level mapped to its probability. They sum to 1.
confidence number
0 to 1. A single peak on one level is high; probability spread across levels is low.

Rules and limits

  • criteria is an ordered array of level descriptions, low end first. It needs at least two levels and takes up to ten.
  • A level’s number is its position in the array, starting at 0.
  • Different distributions produce the same score: 1.0 can mean all the probability sits on level 1, or that it is split evenly between levels 0 and 2. Read probabilities and confidence alongside it.

Reach for a Score when

  • The answers are rungs on one ladder — severity, urgency, frustration, quality, risk.
  • Landing between two levels is informative rather than a problem.
  • You want to combine several ratings with weights you control, instead of asking one vague question about overall quality.

Use something else when

  • The options are not ordered. A Choice is the honest shape for unordered categories.
  • You cannot describe the levels distinctly. Use as many as you can describe and no more — undescribed levels give the model nothing to match against.

The other two

Noul

A Noul is one of the three question types in TypeSafe’s Jev model. It asks a single yes/no question and answers with the probability that the answer is yes, as a number from 0 to 1.

Choice

A Choice is one of the three question types in TypeSafe’s Jev model. It picks exactly one option from a set you define and returns the winner, a probability for every option, and a confidence value.

Ask a Score of your own text

The playground sends whatever you paste to Jev and draws every answer. No account, no key.

Open the playground