results/aggregated_scores_verilator.csv

Model,Agg S2R,Agg MC,Agg VerilogEval S2R,Agg VerilogEval MC,Agg RTLLM,Agg VeriGen
DeepSeek V3.1 Terminus,71.35,75.31,74.22,75.43,61.89,74.2
Kimi K2 Instruct 0905,67.8,71.78,69.72,72.46,61.46,65.63
Gemini 2.5 Flash (Medium),63.27,70.19,67.75,69.55,48.49,76.02
DeepSeek R1-0528,75.83,78.07,77.64,78.78,69.85,71.64
DeepSeek R1,75.78,75.99,78.04,76.42,68.31,72.05
Qwen3 Coder 480B A35B,61.46,58.18,64.01,56.5,53.04,73.45
Llama 3.1 405B,52.08,55.06,54.39,55.1,44.48,54.7
Qwen3 236B A22B,69.17,66.8,74.24,67.55,52.44,60.01
gpt-oss-120b,70.18,74.91,75.45,75.4,52.82,70.46
Qwen2.5 72B,51.72,52.29,52.7,52.44,48.49,50.95
Llama 3.(1-3) 70B,40.06,44.71,39.84,42.69,40.79,63.12
Seed-OSS-36B,67.75,71.92,69.69,72.18,61.35,69.58
QwQ 32B,63.76,41.14,66.09,38.64,56.06,63.91
Qwen2.5 32B,53.2,43.6,54.23,43.05,49.81,48.64
Magistral Small 2506,41.02,23.48,41.25,20.36,40.27,51.89
gpt-oss-20b,63.19,65.93,66.45,67.71,52.45,49.67
StarChat2 15B v0.1,40.2,38.43,37.97,35.85,47.54,61.91
DeepSeek R1 Distill Qwen 14B,22.93,24.82,24.44,24.43,17.96,28.33
Hermes-4-14B-Reasoning,50.57,38.19,55.07,36.12,35.74,57.11
Hermes-4-14B,44.05,28.99,42.69,26.97,48.52,47.41
Qwen3-8B,46.23,48.83,49.8,49.98,34.44,38.32
CodeLlama 70B,34.55,35.64,33.58,33.89,37.75,51.62
DeepSeek Coder 33B,27.93,39.42,23.14,38.14,43.71,51.12
QwenCoder 2.5 32B,45.72,47.29,45.31,46.49,47.07,54.62
DeepCoder 14B,27.06,33.49,27.87,33.39,24.38,34.45
QwenCoder 2.5 14B,39.39,42.39,36.86,41.54,47.73,50.17
SeedCoder 8B,52.04,37.69,52.58,37.24,50.25,41.84
SeedCoder 8B Reasoning,43.41,48.88,47.02,47.51,31.51,61.35
OpenCoder 8B,30.45,36.16,26.96,35.34,41.96,43.63
QwenCoder 2.5 7B,14.91,35.37,7.08,34.02,40.74,47.7
DeepSeek Coder 6.7B,34.14,30.16,30.46,29.39,46.28,37.19
CodeV R1 Distill Qwen 7B,37.26,25.62,33.33,21.67,50.21,61.63
HaVen-CodeQwen,44.57,49.91,45.09,48.07,42.87,66.7
CodeV-QW-7B,21.69,51.2,19.85,50.51,27.77,57.45
RTLCoder Mistral,22.64,29.36,24.75,26.44,15.68,56.02
RTLCoder DeepSeek,38.48,37.5,39.19,35.74,36.14,53.57
OriGen,52.85,52.16,52.72,49.84,53.29,73.3
CodeV-CL-7B,15.32,32.82,13.21,31.06,22.28,48.85
CodeV-DS-6.7B,19.28,47.41,14.23,45.91,35.94,61.05