How much must stay resident?
usual inference≈100%serial weights→0%
Parameter residency, Rp.
Mechanism · one lesson at a time
Watch one expert stream do three different jobs.
Eight blocks stand in for a long model tape. Request A needs E2 and E6. Change the experiment; watch what crosses the boundary.
storage boundary
Q6 decode
↓
MAC
↓
accumulator
↓
MAC
↓
accumulator
Three questions become three metrics
How much useful work per fetched weight?
1 request 1×
8 requests 8×
128 requests 128×
Scan productivity, Ps.
Can compute keep up?
storage ━━━━━━━━━━━━━━━→compute ━→
Consumption balance, η.
Show the detailed FPGA-ish datapath
The measured-trace animation, FIFO, Q6_K decoder, MAC lanes, router mask, and counters live in the Lab.