Decision Models Explained: Why Agents Are Getting Tiny Models That Never Write a Word

Decision models are the newest small-model category to reach agent builders, and four of them shipped in the three weeks before October 5, 2026. TypeSafe AI’s Jev launched on September 15. OpenAI previewed a Decisions API at its September 29 developer event. Cloudflare released Clef and Clef-flash on October 1, and Amazon published Strands Decider 2B the same day. None of them writes a sentence. Each one reads a description of a situation plus a list of allowed answers, and hands back a probability for every answer.

That design targets a narrow and expensive chore: the routing, tool selection and yes-or-no checks that sit between the steps of an agent. Doing those with a frontier chat model works, and it is slow and costly when it happens thousands of times an hour. This article explains how the new models work, compares what the vendors have disclosed, works through the cost math, and sets out how to test one before trusting it. Every performance figure here comes from a vendor or from secondary coverage, and the article says so where it matters.

What a decision model does

Give it a state, which can be text or structured data, and one or more typed questions. A question might ask for a yes or no, a pick from named options, or a score on an ordered scale. The model answers all of them in a single forward pass, returning a probability for each allowed answer. There is no generated text to parse, no malformed JSON to repair, and no sampling step that can wander off the menu.

Cloudflare’s version accepts up to 64 typed questions per request. OpenAI describes its Decisions API as answering developer-defined questions with fixed answer sets, from text or images, with classification, routing and agent next steps as the intended uses. The shared idea is that the answer space is declared at request time. That separates these systems from a conventional classifier, whose labels are baked in during training.

How this differs from asking a chat model for JSON

A chat model produces tokens one at a time. Even with constrained decoding that forces valid output, the result is one sampled answer with no trustworthy sense of confidence. A decision model returns the whole distribution, which unlocks a useful pattern: act automatically when the top answer clears a threshold, and escalate to a larger model or a person when it does not.

Training choices reflect that goal. Cloudflare says it trained Clef with a rank-256 LoRA and a Brier loss term, followed by a reinforcement-learning stage it calls RLCD. A Brier loss penalizes the gap between a stated probability and what actually happened, so it rewards calibration and not only accuracy. Amazon took a more surgical route. Its coverage says AWS replaced the next-word prediction component of a small base model with a pointer component that scores the options supplied, then trained about a million parameters: a rank-16 LoRA plus the scoring head. AWS gives the architecture the name Hobson.

The offerings side by side

Context limits differ as well. Clef takes 65,536 tokens, with text, JSON and up to four images per request, while Cloudflare’s comparison sheet lists Jev at 32,000 tokens of text only. Latency numbers come from the vendors and were measured on different hardware, so only the order of magnitude is comparable.

What the first benchmarks do and do not show

Cloudflare ran its own comparison against Jev on the Jev Decision Index, version 0.2.1. Clef came out ahead on function-calling accuracy (98.5 against 95.8 on BFCL) and on two intent-classification sets, with macro-F1 scores of 94.2 against 79.7 on BANKING77 and 97.4 against 89.3 on CLINC150 with out-of-scope detection. Jev led on the knowledge-heavy tests: GPQA Diamond at 78.3 against 48.0, MMLU-Pro at 82.7 against 65.9, When2Call at 81.0 against 72.4, and BRIGHT retrieval at 47.5 against 45.9.

The split is informative. Routing and classification, where the categories are clear and the input is short, favored the open model. Decisions that lean on world knowledge or reasoning favored the larger hosted system. Both sets of numbers were produced by one party, and Cloudflare’s own page labels them vendor-run. Amazon’s reported score is about 72 percent on the public JevBench set, and the coverage also notes a rival open model scoring higher, which is a reason to treat any claim of first place cautiously.

Where a decision model sits in an agent loop

The natural slots are the junctions: choosing which tool to call, deciding whether a result satisfies the request, routing a message to the right workflow, and reviewing a proposed action before it runs. AWS lists routing requests, selecting tools, evaluating outputs and reviewing agent actions as the intended uses. A request and response might look like this. The field names are invented for illustration and do not match any vendor’s actual schema.

request
  state: "Customer wrote: I was charged twice for order 8841. Plan: pro. Open tickets: 0."
  question route: choose one of [billing, technical, account, other]
  question needs_human: yes or no

response
  route: billing 0.94, technical 0.01, account 0.04, other 0.01
  needs_human: yes 0.31, no 0.69

An agent can act on route at once, because 0.94 clears any sensible threshold, and still pass the case to a person if the needs_human probability climbs. Thresholds are the whole point, and each deployment has to choose its own.

Costs, worked out

Suppose a workflow makes one million decisions a month, and each request carries about 1,500 tokens of state. That is 1.5 billion input tokens. At Jev’s listed rate of $0.042 per million tokens, the bill is $63. Clef-flash at $0.09 comes to $135, and Clef at $0.24 comes to $360. None of the three charges for output tokens, so cost scales with how much state you send, and trimming that state is the cheapest optimization available.

Strands Decider has no per-token price at all. Its cost is whatever hardware and operations you supply. The coverage points out that AWS has not shown self-hosting to be cheaper than Jev’s hosted price once those costs are counted, and it is worth running the numbers for your own volume before assuming it is.

Risks and failure modes

Prompt injection is the first. A decision model reads the state, and the state often contains text written by strangers: a customer message, a web page, an email. Adversarial wording inside that text can push the verdict toward an answer the attacker wants, a risk the Strands coverage itself flags. A decision model should therefore never be the only barrier in front of an irreversible action.

Calibration is the second. Reported figures such as AWS’s statement that answers above 0.9 confidence were right about 95 percent of the time came through secondary coverage and deserve a check against your own data. A model can be well calibrated on its vendor’s test distribution and poorly calibrated on yours.

The third is auditability. With no generated reasoning, a wrong answer arrives without an explanation, so log the inputs and the full probability vectors. A fourth concerns openness. Coverage describes Strands Decider as shipping weights, training data and scripts. A commenter on Cloudflare’s release noted that Clef’s weights are open while its training data and pipeline are not, which is a commenter’s observation and not a verified fact. Apache 2.0 weights allow commercial use either way, but they do not let you reproduce the training.

A practical evaluation plan

1. Pull 300 to 500 real decisions from your logs, each with the correct answer, and keep a quarter of them aside as a held-out set.
2. Measure accuracy for each question type separately, since routing and judgment questions behave differently.
3. Bucket predictions by confidence and compare each bucket’s stated probability with its actual hit rate. That is the calibration check.
4. Pick two thresholds: act automatically above the first, escalate below the second, and review the band in between.
5. Feed in inputs with hidden instructions and see whether the verdict moves.
6. Time end-to-end latency, network included, because a quoted median for the model alone rarely survives contact with production.
7. Compare the cost per thousand decisions with and without hosting overhead, and repeat the whole exercise when the vendor updates the model.

What is not known as of October 5

OpenAI’s Decisions API is in limited preview, with a broader release described as coming in the following days, and no pricing or latency figures have been published. Jev’s parameter count is not disclosed in the sources I could load. No independent benchmark of any of the four exists yet, and I could not reach Cloudflare’s or Amazon’s own announcement pages, so those figures come from secondary coverage. Multilingual behavior and long-context accuracy are untested in everything I found.

Frequently asked questions

What is a decision model? A small model that takes a state and a list of allowed answers, and returns a probability for each answer instead of writing text.

How is that different from a classifier? A classifier has its labels fixed during training. These models accept the question and the allowed answers with every request.

Can a decision model replace a chat model? No. It cannot draft, explain or plan. It acts as the dispatcher that decides what happens next, and a larger model still does the work.

Are they safe to use as guardrails? Only as one layer. Untrusted text in the input can sway a verdict, so pair them with hard limits enforced outside the model.

Sources

• OpenAI, DevDay 2026 recap, September 29, 2026
• Developers Digest, Cloudflare Clef decision models: Jev-compatible, benchmarked and priced
• AI Weekly, Amazon ships Strands Decider 2B, an open-source Jev rival