Decision Models in Hugging Face

Hugging Face large Language Models (LLMs) are exceptionally good at generating text. We use them to answer questions, write code, reason about problems, and so much more.

But not every problem requires generating text. Consider a customer support system receiving the following message: “I was charged twice for my subscription. I need a refund today.” The system needs to determine which team should handle the request, how urgent it is, and whether the customer is requesting a refund. We could ask an LLM to return a JSON object containing these answers. llama cli -hf ggml-org/gemma-4-e4b-it-GGUF:Q4_0 > We have the following customer support ticket, “I was charged twice for my subscription. I need a refund today.”, which team should handle the request, how urgent it is, and whether the customer is requesting a refund. Give the answers in JSON. [Start thinking] … [end thinking] “`json { “team_to_handle”: “Billing”, “urgency”: “High”, “request_refund”: true } But what if the model could directly evaluate the available options and return a probability distribution over them, without generating any text? from transformers import pipeline decide = pipeline(“decision”, model=”autotrust/GEV-26B-Decide”) decision = decide( { “state”: “I was charged twice for my order and want a refund.”, “questions”: { “department”: { “type”: “choice”, “instructions”: “Which team should handle this?”, “criteria”: { “billing”: “Payments and refunds”, “technical”: “Bugs and technical issues”, “sales”: “Pricing and subscriptions” } }, } } ) # { # ‘department’: { # ‘type’: ‘choice’, # ‘choice’: ‘billing’, # ‘confidence’: 0.9974000453948975, # ‘probabilities’: { # ‘billing’: 0.9982666969299316, # ‘technical’: 0.0010246571619063616, # ‘sales’: 0.0007087149424478412 # } # } # } This is the key idea behind decision models. A decision model takes some input context (state), evaluates a question with a set of predefined possible answers (primitives), and returns a structured decision along with probabilities. Instead of asking a model to generate an answer token by token, we ask it to assign probabilities to the answers that our application already understands. TypeSafe AI introduced Jev, describing it as a System One model. The name draws inspiration from the distinction between fast, intuitive thinking (System 1) and slower, deliberative thinking (System 2). The community has converged to the term decision model, because it describes the model’s computational role rather than invoking a particular theory of cognition.