Tic-Tac-Toe, End to End, in MLPL – Fine-Tuned on the Apple GPU
build the game, derive optimal play, train a policy on MLX, prove it by playing – no domain-specific builtins
Table of Contents
Apple-Silicon, true-GPU. The device("mlx") fine-tune block
runs forward, backward, and Adam on the Apple GPU. Not runnable in the
public browser demo (WASM is CPU-only) -- the numbers and chart are a real
run by an MLX-enabled mlpl-repl on Apple Silicon.
The tic-tac-toe demo, entirely in MLPL – no domain-specific builtins. We
build the game engine, derive the optimal policy by alpha-beta minimax,
encode boards, GENERATE the training set by self-play, LoRA-fine-tune a
board -> move policy on the GPU, and prove it learned by playing vs a
random opponent before and after, shown as a waffle. The only "ML
primitives" are chain~/~linear~/~lora~/~apply~/~adam~/~svg; the game is all
MLPL. Blocks share one :session ttt.
The board, and winning, as a matmul
+1 = X (moves first), -1 = O, 0 = empty. The eight lines are rows of a
constant [8,9] incidence matrix M; M @ board is the line-sums, and a
sum of +3 / -3 means X / O won. Winner detection is one matmul.
lines_m = reshape([
1,1,1,0,0,0,0,0,0, 0,0,0,1,1,1,0,0,0, 0,0,0,0,0,0,1,1,1,
1,0,0,1,0,0,1,0,0, 0,1,0,0,1,0,0,1,0, 0,0,1,0,0,1,0,0,1,
1,0,0,0,1,0,0,0,1, 0,0,1,0,1,0,1,0,0], [8, 9])
def u:winner(board) { sums = reshape(matmul(lines_m, reshape(board, [9, 1])), [8]); if gt(reduce_add(eq(sums, 3), 0), 0) { 1 } else { if gt(reduce_add(eq(sums, -3), 0), 0) { -1 } else { 0 } } }
def u:terminal(board) { if gt(abs(u:winner(board)), 0) { 1 } else { eq(reduce_add(abs(board), 0), 9) } }
[u:winner([1,1,1, 0,-1,0, -1,0,0]), u:winner([-1,1,1, -1,1,0, -1,0,0]), u:winner([1,-1,1, -1,1,-1, -1,1,-1])]
1 -1 0
Optimal play, by alpha-beta minimax
u:ab is the minimax value under perfect play (recursion – needs the
per-call scope fix, issue #6); a done flag prunes once alpha >= beta.
u:best_move returns the optimal cell – the training-label oracle.
def u:mx(a, b) { if gt(a, b) { a } else { b } }
def u:mn(a, b) { if gt(a, b) { b } else { a } }
def u:ge(a, b) { if gt(a, b) { 1 } else { eq(a, b) } }
def u:ab(board, player, alpha, beta) { w = u:winner(board); if gt(abs(w), 0) { w } else { if eq(reduce_add(abs(board), 0), 9) { 0 } else { best = 0 - 2 * player; a = alpha; b = beta; done = 0; for c in range(9) { if eq(take(board, 0, c), 0) { if eq(done, 0) { nb = board + player * eq(range(9), c); s = u:ab(nb, 0 - player, a, b); if eq(player, 1) { best = u:mx(best, s); a = u:mx(a, best) } else { best = u:mn(best, s); b = u:mn(b, best) }; if u:ge(a, b) { done = 1 } else { 0 } } else { 0 } } else { 0 } }; best } } }
def u:best_move(board, player) { bv = 0 - 2 * player; bc = 0; seen = 0; for c in range(9) { if eq(take(board, 0, c), 0) { nb = board + player * eq(range(9), c); v = u:ab(nb, 0 - player, 0 - 2, 2); ti = if eq(seen, 0) { 1 } else { if eq(player, 1) { gt(v, bv) } else { gt(bv, v) } }; if ti { bv = v; bc = c; seen = 1 } else { 0 } } else { 0 } }; bc }
[u:ab([1,1,0, 0,-1,0, 0,0,-1], 1, 0 - 2, 2), u:best_move([1,-1,0, 0,1,0, 0,0,0], -1)]
1 2
Encoding, and the on-policy training set
Each cell is encoded from the side-to-move's view as [empty, mine,
theirs] (27 features). We then play ng games where X is random and O
plays optimally, and collect every position O faces with its optimal
move – the on-policy data the model actually needs. (A while loop grows
the dataset; MLPL's for is a fixed-shape scan, while is the general
loop.)
def u:encode(board, mover) { e = eq(board, 0); m = eq(board, mover); t = eq(board, 0 - mover); reshape(concat(concat(reshape(e, [9, 1]), reshape(m, [9, 1]), 1), reshape(t, [9, 1]), 1), [27]) }
def u:rmove(board, s) { argmax(random(s, [9]) * eq(board, 0), 0) }
ng = 90 ; seed = 1 ; g = 0 ; started = 0 ; dx = fill([1, 27], 0) ; dy = fill([1, 1], 0)
while gt(ng, g) { board = fill([9], 0); player = 1; while eq(u:terminal(board), 0) { if eq(player, 1) { mv = u:rmove(board, seed); seed = seed + 1 } else { bm = u:best_move(board, 0 - 1); xr = reshape(u:encode(board, 0 - 1), [1, 27]); yr = reshape(bm, [1, 1]); if eq(started, 0) { dx = xr; dy = yr; started = 1 } else { dx = concat(dx, xr, 0); dy = concat(dy, yr, 0) }; mv = bm }; board = board + player * eq(range(9), mv); player = 0 - player }; g = g + 1 }
N = reshape(take(shape(dx), 0, 0), []) ; X = dx ; Y = reshape(dy, [N])
shape(X)
319 27
Fine-tune on the GPU, and prove it by playing
u:omove is the model's move (encode -> forward -> best legal cell);
u:play_o plays the model (global m) as O vs random X and tallies
[losses, ties, wins]. We measure the untrained LoRA policy, fine-tune the
adapters on the GPU, and measure again. The waffle: top = before, bottom =
after; red loss, gray tie, green win.
def u:omove(board) { logits = reshape(apply(m, reshape(u:encode(board, 0 - 1), [1, 27])), [9]); argmax(logits + (eq(board, 0) - 1) * 1000, 0) }
def u:play_o(n) { loss = 0; tie = 0; win = 0; g = 0; s = 90000; while gt(n, g) { board = fill([9], 0); player = 1; while eq(u:terminal(board), 0) { if eq(player, 1) { mv = u:rmove(board, s); s = s + 1 } else { mv = u:omove(board) }; board = board + player * eq(range(9), mv); player = 0 - player }; w = u:winner(board); if eq(w, 1) { loss = loss + 1 } else { if eq(w, 0 - 1) { win = win + 1 } else { tie = tie + 1 } }; g = g + 1 }; reshape(concat(concat(reshape(loss, [1]), reshape(tie, [1]), 0), reshape(win, [1]), 0), [3]) }
base = chain(linear(27, 128, 0), relu_layer(), linear(128, 9, 1)) ; m = lora(base, 8, 16.0, 0)
before = u:play_o(50)
device("mlx") { experiment "ttt" { train 1800 { adam(cross_entropy(apply(m, X), Y), m, 0.05, 0.9, 0.999, 0.00000001) } } }
after = u:play_o(50)
svg(reshape(concat(before, after), [2, 3]), "waffle")
How this was run
cargo build -p mlpl-repl --features mlx --release MLPL_REPL_CMD="$PWD/target/release/mlpl-repl" \ ./examples/literate/publish.sh examples/literate/tictactoe-finetune.org
Same program as the playground's MLX tic-tac-toe fine-tune demo and
examples/tictactoe-demo.mlpl. The self-play search + GPU fine-tune run a
real Apple-GPU LoRA step; off Apple Silicon device("mlx") falls back to
the (slower, under-fitting) CPU path.