Evidence ledger · September 2026
Choose the kind of evidence. Measured facts are the default; estimates and open ideas never borrow their authority.
Bandwidth and clock models estimate whether a proposed decoder/MAC pipeline can consume a stream without starving. They expose impossible combinations early, but omit routing skew, host contention, conversion overhead, thermals, and real board behavior.
A plausible spreadsheet cell is not measured throughput.
A small or old GPU holding attention, KV cache, and active state—paired with a serial expert engine fed from abundant host memory or storage—may run a MoE model that cannot fit in its VRAM, with better throughput and energy than ordinary CPU offload.
Possible outcome: transfer and compute costs erase the capacity benefit. That would be a useful negative result.
Measured on large12 · three-run medians
This is conventional llama.cpp placement, not the proposed ordered serial engine. Change quant to see why generation speed alone is misleading.
Loading measured placement data…
VRAM is the peak over the complete 1/2/4/8 run. Time is the median for the selected request count and includes 3,840 prompt + 256 generated tokens per request. GPU energy excludes CPU and wall power.
First oversized control · measured · three runs
Gemma-4 26B-A4B Q5_K_M fails all-GPU allocation. Conventional llama.cpp keeps 20/30 layers on GPU. This control describes practical speed and memory; beating it is not the capacity goal.
Loading measured oversized baseline…
Each request is exactly 128 input + 16 generated tokens. Correctness uses distinct deterministic smoke tasks. Energy is NVIDIA-board-only over load plus the full sweep; no requests/kWh claim is possible.
Oversized Gemma · real layer 0 · one run
The router chooses eight experts per activation. The scheduler reads each distinct selected expert once, then applies it to every request that needs it. This is same-quant layer arithmetic, not complete inference.
Loading measured serial-layer data…
Bytes include 6.4% removable stream framing. Timings are scalar Rust on warm files. Accuracy is agreement with a direct Rust GGUF path sharing the quant decoders—not yet llama.cpp logits or coding-task quality.
First complete bounded prototype · measured · three runs
Native GPU attention, KV and shared tensors remain resident. Selected CPU expert pages are mapped on demand and made reclaimable after every expert. Speed is shown as usability context, not a success threshold.
Loading bounded end-to-end data…
45/45 short tasks passed. Peak RSS remains far above one expert; logical expert bytes are not measured physical SSD reads. The 4K+256 qualification and wall power remain open.
Paired executable qualification · measured · two repetitions
The identical Q5_K_M model answered eight Rust tasks under resident and reclaimed expert policies. Generated source was compiled and tested only in offline, resource-limited containers. Select concurrency to compare outcomes and residency.
Loading validation evidence…
All 60 policy responses passed, but this is only eight small functions and two repetitions—not repository-scale agent work. “Logical bytes” are selected tensor demand, not SSD traffic. The repeated physical-I/O qualification appears below.
Cold HDD · measured · three repetitions
Choose concurrency to compare ordinary resident page cache with unconditional expert-page reclamation. Every generated Rust function was compiled and tested.
Loading repeated cold-I/O evidence…
All 180 evaluations passed. Reads remained approximately 4 KiB mmap faults. This measures Linux page-cache reuse, not the proposed repacked sequential stream; device bytes are an idle-device upper bound while process read bytes are attributable.
Physical stream · measured · seven repetitions
The same 169.69 MB validated Gemma expert slice was replayed from HDD and NVMe with synchronous and bounded double-buffer readers.
Loading physical stream evidence…
The 2 MiB bound covers both buffers. Equal byte digests establish identical payloads; activation math was validated separately. This is a layer-0 replay, not end-to-end inference. Parallel SAS and SSD-hot/HDD-cold placement remain unmeasured.
Shared resource service · measured · three cold runs
Independent clients already share Linux physical page-cache reads. The resource scheduler additionally traverses and verifies one bounded stream on behalf of every waiting client.
Loading resource-service evidence…
Byte hashing stands in for a consumer; no expert arithmetic, GPU transfer, KV state, or model quality is measured here. Both policies caused one artifact-sized physical HDD read because Linux page cache already shares concurrent reads.
Install/profile phase · measured metadata and traces
The installer safely inventories an arbitrary supported MoE GGUF and binds its cache proposal to exact model, runtime, hardware, and memory budgets. Granite shows cold start; Gemma shows a trace-derived proposal.
Loading model-profile evidence…
These are proposed placements, not a cache-speed measurement. The next experiment freezes the profile and compares policies on unseen coding work.