The constraint in sparse GPU computation is not the math. It is that cuSPARSE ships one kernel per format, and the best format depends on the matrix. The same SpMM on the same matrix shows a 350x throughput difference between CSR and Blocked-ELL on cuSPARSE. SparseDitto, from researchers at Northeastern and Argonne, addresses this by generating a kernel per matrix rather than per library.
The system is three stages. A lightweight additive model scores established strategies against structural features of the input matrix. An architecture-aware planner proposes candidate designs. Coding and verification agents implement and test them. The result is a kernel tuned to the specific sparsity pattern, operator, and target GPU, covering SpMV, SpMM, and SpGEMM in a unified design loop. The verification agent is load-bearing here: it closes the loop that hand-written sparse compilers leave open, catching implementation errors before the kernel reaches the caller.
The broader pattern is one that hardware teams should track. GPU kernel engineering has been a hand-tuned discipline where performance depends on knowing which formats your library supports and which your workload hits. SparseDitto makes that decision programmable via agents. The same architecture that works for sparse kernels generalizes: wherever there is a large configuration space with workload-dependent optima and a fast oracle (compile, measure, verify), an agentic planner outperforms a static library.
The near-term loser is the assumption that high-performance sparse computation requires a specialist. If an agent can generate a tuned kernel from matrix metadata in the same time a human would spend reading cuSPARSE documentation, the staffing model for inference optimization teams changes in 12-18 months.