SK Hynix engineers published StreamDQ, a near-memory dequantization architecture that moves the weight expansion step for quantized LLMs from the GPU compute die into a logic layer embedded in custom HBM. The constraint being removed is not memory bandwidth. It is the CUDA-core instruction overhead that current GPU dequantization imposes before any compute actually starts.
Weight-only quantization (INT4, INT8, GPTQ, AWQ) is now standard practice for fitting large models into inference infrastructure. The problem is that on a stock GPU, dequantization runs as a kernel on CUDA cores. Before the tensor cores start computing, the GPU spends cycles expanding compressed weights back to FP16 or BF16. StreamDQ pushes that expansion into the HBM stack itself, using near-memory processing logic that can do the conversion in-flight as weights stream toward the compute die. The paper reports that this removes the CUDA-core instruction overhead for dequantization and scales with model size in a way that software-side dequantization cannot, because the bottleneck moves from compute cycles to memory bandwidth, which is the direction HBM already optimizes for.
The architectural implication is that quantized inference efficiency is no longer purely a software problem. The HBM stack is becoming a co-processor for model serving, not just a memory array. Teams designing next-generation inference ASICs now have a concrete reference point for where the compute-memory boundary can move: not at the HBM4 interface, but inside the stack itself. The longer-term signal is that memory vendors who can differentiate on compute-near-memory features will have an argument for being in the BOM conversation for inference cards, not just as a commodity DRAM supplier. SK Hynix publishing this architecture publicly, rather than shipping it quietly in a product, suggests they are establishing the pattern rather than protecting it.