Serial Parameter Machineemufpga research notebook

Question 00 · capacity before speed

Can a ten-year-old server run a model that doesn’t fit in GPU memory?

This experiment treats immutable model weights more like a tape than RAM: move the needed MoE experts past compute, while the small GPU keeps attention, KV cache, and active state.

Conventional model residency versus serial parameter flowThe usual approach requires all weights to fit near the GPU. The experiment streams expert weights from cheap storage through a small buffer and compute engine while GPU state remains resident.THE USUAL WAYGPU / VRAMall weights must fit hereORTHE EXPERIMENTcheap storagetiny bufferdecode + MACCPU / FPGA-ishGPU state
Can we trade memory capacity for sequential bandwidth?Not “is tape faster than VRAM?”—can reused hardware do useful work that otherwise needs a larger GPU?

MAC does the repeated model arithmetic. FPGA means reconfigurable digital logic.

What is this?

A software and hardware experiment in serial parameter execution, especially route-selected MoE experts.

→

What can we measure?

Correctness, bytes fetched, residency, reuse, stalls, latency, throughput, memory, and eventually whole-system energy.

→

What are we trying to find out?

Whether a small old GPU plus streamed experts beats CPU/RAM offload for a model too large for that GPU.

Watch one layer move →See what we actually know →