The binding constraint for large MoE LLM inference is not compute -- it is HBM capacity. Expert weights for frontier MoE models cannot fit in the HBM provisioned near the GPU, so either you pay for more HBM or you serialize expert fetches. This paper adds a third option: route expert weights over two independent paths simultaneously, one direct from high-bandwidth flash (HBF) to the GPU, and one relayed through the HBM base die. Whole experts are assigned to one path or the other, transfers proceed concurrently, and the combined bandwidth exceeds what either path delivers alone. Early expert determination -- predicting which experts will activate ahead of execution -- lets HBF read latency overlap with the preceding compute step.
The architecture also separates immutable expert weights from mutable KV cache in HBF management, which prevents the two traffic classes from interfering with each other during prefetch. The evaluation uses an event-driven continuous-batching simulator with empirically measured GPU compute latencies -- not a purely analytical model -- which makes the throughput projections more credible than most memory architecture papers at this stage.
The shift this represents: flash memory is moving from cold-tier spill storage to an active inference memory tier with its own direct interconnect to the GPU. That changes the hardware design question for AI inference accelerators from "how much HBM can we stack" to "how do we architect the flash path." Vendors whose AI server designs assume HBM is the only fast memory tier proximate to compute will need to revisit that assumption. The HBM supply chain constraint is real and persistent; this paper maps one engineering path around it.