There is no agreed quality bar for LLM-generated Verilog. Teams evaluating whether to put LLM code generation into their RTL flows are working from vendor demos and pass@k metrics that do not reflect what actually happens when generated code hits Verilator or Icarus. This paper ships a reproducible benchmark that changes that: 50 curated RTL tasks across combinational, sequential, FSM, and mixed designs, validated through the full compile-and-simulate pipeline, with an iterative refinement loop that includes waveform analysis, formal equivalence checking, and AST-based repair. Across 1,610 total runs on three open-source models, the pipeline improves syntax validity from 0% to 70.43% average and gets simulation pass rate to 51.8%. The benchmark and dataset are open source.
The number that cuts through the marketing: 51.8% functional correctness after iterative refinement. Nearly half of generated RTL does not simulate correctly even after the pipeline has applied waveform feedback, formal equivalence checks, and AST repair. That is not a fatal result for LLM-assisted RTL; it is a calibration number. It tells a team what they are actually deploying: a co-pilot that gets half of leaf-level RTL tasks right on the first synthesizable attempt, not a replace-the-engineer tool. The second number worth keeping: TinyLlama at 1.1B parameters achieves 80% syntax validity and functional correctness comparable to the 8B Llama-3 model. Model scale does not linearly predict HDL suitability. The binding variable appears to be training-data alignment with synthesizable RTL patterns, not raw parameter count.
Teams deciding whether to integrate LLM RTL generation into their flow now have a reproducible baseline to run their own models against, rather than relying on vendor benchmarks designed to show the ceiling. The practical action is to clone this benchmark, run your model of choice, and compare the resulting sim-pass rate against 51.8% before committing engineering time to prompt engineering and workflow integration. A model that cannot beat the benchmark average is probably not worth the integration cost.