GPU tensor cores were bolted onto a threading model designed for scalar workloads. The CUDA thread-register binding (each thread owns a fixed slice of the register file) made sense when threads ran scalar ops. It does not hold up when the schedulable unit is a tensor operation that spans multiple threads and benefits from variable parallelism granularity. A paper accepted to ESWEEK 2026 identifies fixed parallelism and coarse-grained scheduling as the two bottlenecks that survive after the industry moved from Ampere's register-based operand supply to Hopper and Blackwell's memory-based supply.
The proposed execution model decouples thread identity from register allocation. Tensor operations become first-class schedulable units that the hardware assigns registers to independently of the CUDA thread hierarchy. The scheduler can tune parallelism granularity to the workload shape: small batches get narrower allocation, large matrix operations get wider allocation, without routing everything through a fixed thread-to-register table.
The design implication: if you are building inference silicon and are tempted to replicate CUDA thread semantics for compatibility, this paper makes the cost explicit. The thread model is the constraint you are inheriting, not the performance floor you are starting from. Teams building custom silicon for specific workloads (edge inference, MoE routing, long-context attention) now have a concrete argument for dropping CUDA thread semantics at the ISA level and treating register allocation as a scheduler problem rather than a programmer-visible abstraction. The standard objection is CUDA compatibility, which is a real constraint for anyone with an existing software stack. The teams without legacy CUDA to protect are 18 months ahead of the teams that are waiting for the constraint to be lifted upstream.