Mixture-of-Experts models are the shape of frontier LLM inference. The binding constraint on MoE performance is not compute; it is data movement. Every expert activation requires moving model parameters, and the penalty compounds at batch scale. DynaNDE addresses this by making expert scheduling hardware-aware across the NPU-NDP boundary rather than treating NPU and near-data processing as fixed-policy execution units.
The framework builds an analytical performance model that captures hardware heterogeneity, data-movement costs, and communication-computation overlap for NPU-NDP execution. Guided by that model, DynaNDE assigns per-layer expert scheduling across NPU and NDP accounting for expert-level concurrency. A reuse-aware runtime avoids moving parameters again when experts already reside in NPU memory. Compared to the prior state-of-the-art NPU-NDP MoE serving framework: 2.6x prefill speedup, 2.2x decoding speedup.
Near-data processing as an architecture for AI inference is no longer a research curiosity; commercial NPU stacks now ship with NDP-capable memory. DynaNDE is the first scheduler that treats the NPU-NDP boundary as a programmable resource with a tractable analytical model rather than a fixed execution policy. Infrastructure teams deploying MoE workloads at scale on NPU hardware should run this against their current dispatch logic before the next hardware procurement cycle. The headroom is 2x+ without touching silicon.