MoE Microscope live demo

Loading

What you are looking at

Where it comes from

Recordings: fixtures/recordings/*.json in the moe-microscope repository. Each lesson page there explains the experiment, its diagrams, and its measured numbers: experiments, findings.

Run the training yourself

The fixtures this page plays back are produced by the repository's just recipes on a laptop with the adjacent sw-MLPL checkout. Each recipe trains the model natively, writes the fixture, and the same recipe without write checks it in the gate.

git clone https://github.com/sw-ml-study/moe-microscope
cd moe-microscope        # needs ../sw-mlpl built, just, mlplunit
just dense write         # DN01: train 300 epochs (about 16 s), diagrams and row
just moe write           # MX01: four experts, top-1 (about 36 s)
just moe2 write          # MX02: top-2 (about 36 s)
just recur write         # RC01: one block applied R=1..4 times (minutes)
just rm write            # RM01: the mixture block applied R=2,3 times (minutes)
just engram write        # EG01: four table sizes plus the builtin form (minutes)
just re write            # RE01: recurrence, routing, and Engram together (minutes)
just recordings write    # recapture the seven recordings over mlpl-serve
just walkthrough write   # retrain 40 epochs and rewrite the walkthrough fixture
just serve-learn         # compose this page locally and serve it
just check               # the whole gate, including these fixtures