Tenstorrent is launching the Galaxy Blackhole server: a 6U unit packing 32 Blackhole chips, delivering 23 PFLOPS in Block FP8 with a unified prefill and decode architecture. EE Times tested it ahead of launch -- 255 tokens per second per user in chatbot-style workloads, and up to 350 tokens/second in Blitz Mode for agentic inference workloads. Time to first token in Blitz Mode is under 4 seconds for DeepSeek-671B.
The no-disaggregation stance is a deliberate counter-positioning move. The dominant infrastructure trend for large-scale LLM inference is to split prefill and decode across separate hardware pools -- often different accelerator types -- because their compute profiles are fundamentally different. Tenstorrent is betting that scale-out with uniform Blackhole chips in Galaxy clusters can match or beat that efficiency without the operational complexity of heterogeneous disaggregated systems.
Jim Keller's framing -- "we do something no one else does, where we can hook up a lot of medium-performance chips together" -- is honest about what the architecture actually is: a scaling play, not a single-chip performance play. That is a different risk profile than Nvidia (dominant single-chip perf) or custom silicon (domain-locked). Whether it holds up at hyperscaler workloads is unproven, but the token-throughput numbers EE Times independently validated are not marketing slides.
The real test will be whether typical clusters at 4-32 Galaxies can handle the memory bandwidth requirements of very large models. The Blackhole's on-chip memory and interconnect are the constraint that does not show up in PFLOPS metrics. Watch for enterprise adopters and the pricing story.