Condition: Guided — models given the questions plus expert domain guidance
Figures shown: raw response value from every individual run (10 per model), as a distribution — not accuracy

Page generated: 2026-08-16
Data (model runs) generated: 2026-04-11
Scoring inputs: ground truth cc2ed8f5, evaluate.py 845b70e9

Signed Error from Ground Truth: con (Q9) vs e (Q11) — Expert Guided

Each bar = mean model response − GT for that page across 10 runs. Zero line = ground truth. Bars above = over-count, bars below = under-count. Blue = Q9 (con count), Red = Q11 (e conjunction count). Both at 50% transparency so overlaps are visible. Hover for details.

Claude

GPT

Gemini 2.5

Gemini 3.1

Claude (thinking)