The concise hello: a typed decision model in thirty-eight lines
what "Jev in 25 lines of Python" looks like when the model is yours, and what that comparison hides on both sides

A widely-shared post shows a "typed decision model" in about twenty-five lines of Python. Its shape is:

prompt -> a pretrained causal language model
       -> the logits at one position
       -> keep the three that correspond to the tokens A, B and C
       -> softmax those three
       -> a probability per choice, with no token generated

It is a good demonstration and the arithmetic is right. What the line count leaves out is everything underneath: a 0.6B-parameter model to download, a tokenizer, a runtime to host them, and the pretraining corpus that made those logits mean anything. The three choices also have to be smuggled in as the tokens A, B and C, which is why practitioners in that discussion report permuting the letters and checking how much probability mass the allowed tokens hold: the model is being asked a question in a vocabulary that was never about this question.

This page shows the same interface with the model in your hands – and then spends as much space on what this version hides, because a comparison that only counts the other side's costs is not worth reading.

The model is the arrays this file declares, and the answer's domain is named in the data: column 0 is phishing. There is no tokenizer to mislead and no letters to permute. Inference is five lines:

h    = mean of the embedding rows this message's known words address
kind = softmax(apply(os_kind, h))        -- Choice over three categories
asks = sigmoid(apply(os_asks, h))        -- Noul: does it want credentials?
urg  = softmax(apply(os_urgency, h))     -- Scale over three levels
                                         -- and no token was generated

Everything else in the file is the part the Python version cannot do at all: train the thing.

Twelve messages and their three labels, in a separate file passed in on the command line – because the program this is measured against does not carry a corpus either; what it carries instead is a pretrained model that already saw one. Tangling this document writes it:

{
  "kinds": ["phishing", "legitimate", "spam"],
  "levels": ["low", "medium", "high"],
  "texts": [
    "your password expires today, sign in here",
    "payroll needs your bank details now",
    "verify your account or it will be locked",
    "confirm your login to keep your files",
    "the standup moved to ten tomorrow",
    "here are the notes from monday",
    "your package was delivered downstairs",
    "can you review the draft by friday",
    "you have won a free cruise today",
    "cheap watches direct from the factory",
    "lose twenty pounds with this trick",
    "make money from home this week"
  ],
  "kind":        [0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2],
  "credentials": [1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0],
  "urgency":     [2, 2, 2, 2, 1, 0, 0, 1, 0, 0, 0, 0]
}

The module comment, and the include. That one line is not free, and the section after the program counts what it costs.

# The whole idea on one screen, annotated in docs/literate/concise-hello.org: a typed decision model that trains itself, then answers three bounded questions about a sentence it has never seen. The comparison it exists for is the widely-shared "Jev in 25 lines of Python", whose twenty-five lines sit on top of a downloaded language model and its tokenizer; there is nothing underneath this one but the arrays it declares. The corpus is not in the file -- it is passed in on the command line, the way that program's pretraining data is not counted either: mlpl-repl concise-hello.mlpl -- demos/typed-decisions/messages.json. The longer demos/typed-decisions/typed-decisions.mlpl is this program plus what a claim needs: a hundred and twenty messages, held-out splits, and calibration.
include "../../lib/choice_model.mlpl";

The first line is the whole of the data handling: the corpus arrives as a path on the command line, and parse_json turns it into arrays. Nothing about the messages is written into this file.

The four bindings after it exist because of a language rule: a record field may not be read inside grad, so anything the loss touches is pulled into a plain array first. (One call deep the interpreter used to report that the loss did not depend on the parameter; it now names the field, a fix that landed upstream while this was being written.)

Then the model: one embedding, and three heads built with the Model DSL's linear, which carries its own initialization and bias. Eight dimensions of state, three columns for the Choice, one for the Noul, three for the Scale – 11 by 8 plus 8 by 7 plus 7, about 600 numbers once the vocabulary is counted.

os_data = unwrap(parse_json(unwrap(read_text(unwrap(list_get(args(), 0))))));
os_vocab = u:text_vocab(os_data.texts, 12);
os_f = u:cm_featurize_vocab(os_data.texts, os_vocab, 12);
os_ids = os_f.ids;
os_wm = os_f.wmask;
os_cred = reshape(os_data.credentials, [list_len(os_data.texts), 1]);
os_y = os_data.kind;
os_u = os_data.urgency;
os_E = param[os_vocab.size + 1, 8];
os_E = randn(3, [os_vocab.size + 1, 8]) * 0.1;
os_kind = linear(8, 3, 4);
os_asks = linear(8, 1, 5);
os_urgency = linear(8, 3, 6);

The encoder, entire. Look up a row per known word, take their mean. A word the corpus never used has no row to look up and contributes nothing – which is worth remembering when you read the last line of the output below.

def u:os_state(ids, wmask) {
  "The shared state: the mean of the embedding rows this message's known words address.";
  reduce_add(gather_rows(os_E, ids) * wmask, 1)
}

The loss adds one term per question, and the encoder underneath is pulled by all three at once, which is what makes it a shared representation rather than three private ones.

def u:os_loss(ids, wmask, cat, cred, urg) {
  "Cross-entropy for the Choice, binary cross-entropy for the Noul, cross-entropy for the Scale, added: three questions, one encoder, one number to descend.";
  h = u:os_state(ids, wmask);
  p = sigmoid(apply(os_asks, h));
  cross_entropy(apply(os_kind, h), cat) + cross_entropy(apply(os_urgency, h), urg) + (0 - mean(cred * log(p + 0.000000001) + (1 - cred) * log(1 - p + 0.000000001)))
}

Three hundred Adam steps, about two tenths of a second.

train 300 {
  adam(u:os_loss(os_ids, os_wm, os_y, os_cred, os_u), [os_E, os_kind, os_asks, os_urgency], 0.1, 0.9, 0.999, 0.00000001)
};

Then it interrogates itself on six sentences, none of which are in the corpus, and prints how many features it recognized in each. That count is the honest part: it is the difference between an answer and a guess, and the program prints it whether or not it flatters the result.

os_probes = str_split("hr asks you to confirm your password|please re-enter your credentials to avoid suspension|your mailbox is full, log in to restore delivery|the invoice you requested is attached|can we move tomorrow's standup|urgent: wire the funds before close of business", "|");
os_i = 0;
while lt(os_i, list_len(os_probes)) {
  os_m = u:text_lg(os_probes, os_i);
  os_new = u:cm_featurize_vocab(str_split(os_m, "\n"), os_vocab, 12);
  os_h = u:os_state(os_new.ids, os_new.wmask);
  os_a = u:choice("what kind of message is this?", os_data.kinds, reshape(apply(os_kind, os_h), [3]));
  os_b = u:noul("does it ask for credentials?", take(reshape(apply(os_asks, os_h), [1]), 0, 0));
  os_c = u:scale("how urgent is it?", os_data.levels, reshape(apply(os_urgency, os_h), [3]));
  print(take(os_new.counts, 0, 0), "known ", u:decision_label(os_a), floor(os_a.confidence * 1000) / 1000, " credentials", floor(os_b.p * 1000) / 1000, " urgency", u:decision_label(os_c), " <-", os_m);
  os_i = os_i + 1
};
print("")

Both files are published beside this page: concise-hello.mlpl and messages.json. With sw-MLPL's interpreter (measured against mlpl-repl 0.22.0) on your path:

mlpl-repl concise-hello.mlpl -- messages.json

In the repository it is just concise-hello, and scripts/check-typed-decisions runs it in the gate and fails if the output below changes. Unlike the four-stage program it will not run in a browser: a corpus passed in means read_text and args, and the Live Editor has no file system.

6 known  phishing 0.999  credentials 0.999  urgency high    <- hr asks you to confirm your password
2 known  phishing 0.999  credentials 0.999  urgency high    <- please re-enter your credentials to avoid suspension
3 known  phishing 0.999  credentials 0.999  urgency high    <- your mailbox is full, log in to restore delivery
2 known  legitimate 0.999  credentials 0  urgency medium    <- the invoice you requested is attached
2 known  legitimate 0.999  credentials 0  urgency medium    <- can we move tomorrow's standup
1 known  legitimate 0.999  credentials 0  urgency medium    <- urgent: wire the funds before close of business

Read the last line before the first five. "Urgent: wire the funds before close of business" is a textbook business-email-compromise request, and this model calls it legitimate at 0.999. The one feature it recognized was the word "the". Everything else in that sentence – wire, funds, urgent, business – is absent from twelve training messages, so the state it pooled was one row, and the head turned that into a confident answer anyway.

That is not a bug in the arithmetic. It is what a twelve-example model is, and it is why the confidence numbers on this page should not be believed.

The program says include "../../lib/choice_model.mlpl" and then claims to be thirty-eight lines. That include is not free, so here is the bill, produced by the same reachability analysis the gate's bundler runs – it walks the call graph from the program's top-level statements and counts only what can actually be reached:

So the honest figure is 320 lines of MLPL – 46 in the file and 274 it reaches – all of it in this repository and all of it readable. Rather more than half of what it reaches is the decision contract, which is what makes the output a typed decision rather than a bare array; the featurizer accounts for most of the rest.

The comparison, then, is not 38 against 25. It is:

Neither number is impressive on its own. The difference is what you can open.

Isn't this just logistic regression over a bag of words?

Essentially, yes – a mean-pooled embedding with three linear heads is a generalized linear model with a learned feature map, and nothing about it is new. The claim here is not novelty of architecture. It is that the interface is worth having: bounded domains named in data, several typed questions from one pass, probabilities exposed rather than prose, and the branch owned by code you can read. If that shape is best served by a small linear model, that is a point in its favour, not against it.

Then why not three lines of scikit-learn?

For this task, scikit-learn would be shorter and better tuned, and if your goal is a classifier you should use it. Two things it would not give you: the same language training the model and running it (this file is also what the browser runs, in the four-stage version), and a typed decision object – a Choice that carries a distribution, a selection, a confidence and a margin; a Noul that is one probability; a Scale that has an expectation because its labels are ordered. The contract is the product.

Your model says 0.999 about everything. That number is meaningless.

Correct, and the program prints the evidence against itself: the same 0.999 for six known features and for one. Twelve examples, three hundred steps, no regularization and no held-out data will do that. The four-stage document is where it is measured on 120 messages and 24 it never saw (0.750 accurate while claiming 0.912) and then corrected with one fitted temperature (claim 0.652, expected calibration error 0.162 to 0.097). Nothing on this page is a calibration claim.

Isn't it just keyword matching with extra steps?

At twelve examples, largely yes, and the failure above is what that looks like. A bag of words with a learned embedding can only generalize along words it has seen; "wire the funds" shares nothing with the training set but a determiner. The architecture is not what fixes that – more data is, and the larger version still gets only 0.750 with ten times as much. Treat the small number as the honest ceiling of a small corpus.

The comparison is unfair: their twenty-five lines answer any question you can phrase; yours answers three it was trained for.

Correct, and the asymmetry cuts against this page. It has also been measured rather than argued: the four-stage document runs the same 24 held-out messages through local models by the same bounded-choice method, and a three-billion-parameter model answers them better with no training at all (0.792 against 0.750), while a thirty-one-billion one gets all 24 right. If your categories are spam and phishing, a pretrained model already knows those words and you should use one.

What a trained typed decision model has instead is 11,655 parameters against three billion, a decision in microseconds rather than a third of a second, weights you can print, and an answer domain named in a data file. Where it actually earns its place is a question no pretrained model has seen – choosing among the 35 decomposition rules of a 1966 script, say, rather than among words the internet has discussed at length.

How do I know the numbers on this page are real?

Every block above tangles into demos/typed-decisions/concise-hello.mlpl byte for byte, and just check fails if it does not. The gate runs the program and fails if the output in "What it prints" changes. The line counts in the table come from the same script the bundler uses, not from a hand count. Nothing here was typed into prose from memory.

• Typed decisions: the hello world – the same three heads over 120 messages, with held-out splits and the calibration measurement this page deliberately does not make.
• The live demo – the full model, in your browser, with the trace open.
• The repository; this document's source is docs/literate/concise-hello.org, and it tangles both concise-hello.mlpl and its corpus byte for byte.