Hybrid LLMs (GQA + SSM + MoE) broke existing near-memory processing designs in two separate ways. First, MoE models with hundreds of experts exceed DRAM capacity at 10nm-class nodes because the conventional 6F2 cell is at its physical scaling limit. Second, the arithmetic intensity swings from memory-bound SSM layers to compute-intensive GQA layers to expert-routing MoE layers within a single model, and no single NMP arithmetic unit can cover the full range efficiently. NOVA addresses both simultaneously and was accepted at MICRO 2026.
On the technology side, NOVA replaces the 6F2 cell with a 4F2 vertical channel transistor DRAM combined with a peri-over-cell structure, getting approximately 2x memory density at iso-area. On the architecture side, it repurposes the peripheral die to host a two-tier processing stack: Tier-1 on the peri-die handles low-to-mid arithmetic intensity operations (SSM, memory-bound expert routing), Tier-2 on the base die handles mid-to-high operations (GQA). Parallel execution across tiers matches compute resources to the actual intensity of each layer rather than forcing heterogeneous workloads through a single bottleneck.
On Nemotron3, Falcon-H1R, and Qwen3 benchmarks, NOVA averages 4.5x throughput, 69.8% lower end-to-end latency, and 5x better energy efficiency versus a GPU baseline, with 3.9% area overhead and no capacity loss. The numbers are credible because the technology wall and architecture wall were addressed together; prior NMP designs that only touched one dimension consistently degraded on the other.
The implication for inference silicon teams is that DRAM cell geometry is now a first-class design variable for LLM serving, not a foundry constraint to accept passively. Vendors still sizing inference clusters around GPU HBM capacity without evaluating 4F2-class NMP alternatives will face a compounding cost disadvantage as MoE model sizes grow.