Falsifiable next step

Now test the reason to build it.

The next experiment must use a model that cannot fit in the test GPU’s VRAM and compare against the practical fallback: CPU/RAM offload.

Claim: old hardware can run an oversized MoE correctly

capacity✓ short contractall-GPU fails; bounded hybrid generates

correctness✓ 45/45 smokereal same-quant short tasks passed

reliability?coding tasks and reference logits remain

usabilitymeasured limitsreport speed, concurrency, memory, and energy

Experiment 05Oversized-model crossover

Control

SSD→RAM↔Xeon↔small GPU

Current llama.cpp-style CPU/RAM offload.

Serial candidate

SSD→RAM→expert stream→GPU state

First emulate the stream on CPU/GPU; only then substitute FPGA hardware.

Pick a routed MoE quant larger than available VRAM.
Fix prompt set, context, output length, concurrency, and correctness gate.
Measure time-to-first-token, tokens/s, peak RAM/VRAM, bytes moved, and wall joules.
Repeat across concurrency until either approach crosses over or saturates.

Decision rule written before the run

Success

A model that fails resident GPU allocation completes end-to-end with same-quant correctness and bounded GPU memory.

Failure

The hybrid cannot generate correctly or exceeds the small GPU memory budget.

Mixed

Capacity works correctly but has severe speed, concurrency, memory-tier, energy, or hardware-generation limits. Report them without erasing the capacity result.

Open the detailed Lab instrument →