Condition: Naive vs Guided — both conditions, scored identically
Figures shown: mean accuracy across all questions and all 10 runs, per page, naive vs guided
Page generated: 2026-08-16
Data (model runs) generated: 2026-04-11
Scoring inputs: ground truth cc2ed8f5, evaluate.py 845b70e9
Pages: 39 | Models: Claude, GPT, Gemini 2.5, Gemini 3.1, Claude (thinking) | Runs per condition: 10
Combined mean accuracy is the average across all 5 models × all 22 questions × 10 runs per page. Δ = guided − naive.
Click a sort button below to reorder the x-axis. Click a legend entry to toggle a series. Hover any bar for details.
Sorted by combined mean accuracy (lowest first). For each model, the left cell is the naive mean and the right cell is the guided mean. Mean N / Mean G = combined means across all models. Δ = guided − naive in percentage points.
| Rank▲ | Page▲ | Image | Claude | GPT | Gemini 2.5 | Gemini 3.1 | Claude (thinking) | Combined | Structural density | Visual density | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Naive | Guided | Naive | Guided | Naive | Guided | Naive | Guided | Naive | Guided | Mean N | Mean G | Δ | |||||
| 1 | 2 | 1767/8 02 01 n1 | 46% | 47% | 34% | 30% | 48% | 55% | 46% | 53% | 52% | 47% | 45% | 46% | +1.3 | 65 | 2,695 |
| 2 | 4 | 1767 MV 02 01 n1 | 42% | 43% | 46% | 49% | 40% | 56% | 35% | 58% | 46% | 47% | 42% | 51% | +9.0 | 46 | 1,596 |
| 3 | 11 | 1768 05 08 n4 | 39% | 37% | 22% | 44% | 65% | 59% | 46% | 68% | 39% | 44% | 42% | 50% | +8.2 | 118 | 2,633 |
| 4 | 37 | 1770 05 28 n2 | 48% | 45% | 22% | 35% | 49% | 62% | 64% | 52% | 47% | 47% | 46% | 48% | +2.3 | 108 | 2,502 |
| 5 | 23 | 1768 MV 01 09 | 41% | 45% | 41% | 33% | 54% | 55% | 47% | 47% | 51% | 60% | 47% | 48% | +1.5 | 49 | 1,989 |
| 6 | 22 | 1768 MV 01 02 | 39% | 44% | 38% | 35% | 49% | 70% | 54% | 68% | 43% | 51% | 45% | 53% | +8.9 | 51 | 2,366 |
| 7 | 24 | 1768 MV 01 16 n1 | 50% | 48% | 35% | 45% | 44% | 54% | 59% | 61% | 47% | 53% | 47% | 52% | +5.4 | 32 | 1,656 |
| 8 | 8 | 1768 05 08 n1 | 46% | 31% | 38% | 43% | 60% | 55% | 72% | 69% | 41% | 45% | 51% | 49% | −2.6 | 99 | 2,691 |
| 9 | 10 | 1768 05 08 n3 | 45% | 40% | 28% | 43% | 71% | 67% | 58% | 60% | 50% | 45% | 50% | 51% | +0.3 | 97 | 2,480 |
| 10 | 34 | 1770 05 21 n2 | 43% | 54% | 37% | 37% | 58% | 54% | 60% | 51% | 54% | 60% | 50% | 51% | +1.1 | 56 | 1,778 |
| 11 | 19 | 1768 11 07 | 50% | 48% | 51% | 51% | 42% | 53% | 51% | 60% | 55% | 54% | 50% | 53% | +3.5 | 53 | 2,308 |
| 12 | 27 | 1768 MV 02 20 n1 | 54% | 45% | 44% | 46% | 62% | 49% | 58% | 58% | 52% | 48% | 54% | 49% | −4.8 | 67 | 2,629 |
| 13 | 6 | 1768 03 28 | 57% | 50% | 45% | 47% | 61% | 57% | 55% | 45% | 51% | 55% | 54% | 51% | −3.1 | 46 | 2,597 |
| 14 | 16 | 1768 05 16 | 42% | 45% | 33% | 35% | 64% | 58% | 75% | 61% | 61% | 59% | 55% | 51% | −3.8 | 35 | 1,935 |
| 15 | 28 | 1768 MV 02 20 n2 | 47% | 46% | 48% | 46% | 61% | 73% | 54% | 63% | 46% | 51% | 51% | 56% | +4.5 | 36 | 1,614 |
| 16 | 36 | 1770 05 28 n1 | 54% | 61% | 43% | 41% | 54% | 56% | 62% | 57% | 52% | 55% | 53% | 54% | +1.0 | 73 | 2,585 |
| 17 | 26 | 1768 MV 01 16 n3 | 43% | 48% | 45% | 43% | 52% | 58% | 60% | 68% | 50% | 68% | 50% | 57% | +6.8 | 36 | 1,818 |
| 18 | 1 | 1767 06 01 | 50% | 43% | 38% | 38% | 57% | 69% | 73% | 73% | 58% | 61% | 55% | 57% | +1.5 | 56 | 2,136 |
| 19 | 21 | 1768 12 05 | 60% | 65% | 42% | 43% | 49% | 57% | 67% | 65% | 50% | 67% | 54% | 59% | +5.9 | 48 | 1,857 |
| 20 | 12 | 1768 05 08 n5 | 51% | 44% | 37% | 43% | 77% | 77% | 66% | 77% | 63% | 60% | 59% | 60% | +1.2 | 72 | 1,710 |
| 21 | 38 | 1770 05 28 n3 | 57% | 54% | 44% | 45% | 79% | 83% | 65% | 62% | 60% | 59% | 61% | 61% | −0.5 | 38 | 1,915 |
| 22 | 5 | 1767 MV 02 01 n2 | 67% | 59% | 46% | 55% | 35% | 49% | 79% | 71% | 74% | 82% | 60% | 63% | +2.8 | 20 | 1,218 |
| 23 | 9 | 1768 05 08 n2 | 54% | 42% | 48% | 52% | 79% | 76% | 69% | 79% | 62% | 60% | 62% | 62% | −0.5 | 51 | 1,614 |
| 24 | 20 | 1768 11 28 | 50% | 60% | 41% | 53% | 60% | 68% | 77% | 73% | 69% | 74% | 59% | 66% | +6.2 | 39 | 2,157 |
| 25 | 33 | 1770 05 21 n1 | 62% | 58% | 58% | 53% | 57% | 54% | 81% | 82% | 57% | 65% | 63% | 62% | −0.5 | 28 | 1,939 |
| 26 | 14 | 1768 05 09 n2 | 63% | 59% | 58% | 60% | 68% | 75% | 71% | 61% | 67% | 67% | 65% | 64% | −1.1 | 38 | 1,838 |
| 27 | 35 | 1770 05 21 n3 | 45% | 50% | 48% | 59% | 79% | 75% | 81% | 92% | 69% | 71% | 64% | 69% | +5.1 | 19 | 1,571 |
| 28 | 17 | 1768 08 01 | 61% | 65% | 52% | 57% | 69% | 57% | 85% | 81% | 69% | 80% | 67% | 68% | +0.4 | 43 | 1,530 |
| 29 | 32 | 1770 03 13 | 59% | 57% | 60% | 60% | 78% | 77% | 73% | 72% | 81% | 81% | 70% | 70% | −0.4 | 31 | 1,800 |
| 30 | 25 | 1768 MV 01 16 n2 | 50% | 54% | 58% | 68% | 78% | 74% | 85% | 86% | 73% | 80% | 69% | 73% | +4.0 | 27 | 1,102 |
| 31 | 30 | 1769 05 08 n2 | 62% | 65% | 59% | 55% | 80% | 81% | 79% | 90% | 72% | 79% | 70% | 74% | +3.8 | 24 | 1,305 |
| 32 | 29 | 1769 05 08 n1 | 68% | 74% | 67% | 63% | 71% | 89% | 90% | 90% | 81% | 88% | 75% | 81% | +5.3 | 25 | 1,438 |
| 33 | 31 | 1770 03 05 | 67% | 66% | 71% | 68% | 83% | 79% | 90% | 87% | 85% | 88% | 79% | 78% | −1.7 | 19 | 1,897 |
| 34 | 13 | 1768 05 09 n1 | 72% | 67% | 70% | 65% | 73% | 94% | 100% | 95% | 90% | 88% | 81% | 82% | +0.9 | 10 | 782 |
| 35 | 18 | 1768 09 12 | 73% | 77% | 66% | 71% | 86% | 83% | 95% | 87% | 91% | 94% | 82% | 82% | ±0.0 | 20 | 1,225 |
| 36 | 15 | 1768 05 09 n3 | 85% | 76% | 65% | 71% | 74% | 81% | 96% | 99% | 87% | 91% | 82% | 84% | +2.0 | 7 | 1,431 |
| 37 | 3 | 1767/8 02 01 n2 | 93% | 84% | 43% | 56% | 85% | 90% | 95% | 100% | 95% | 96% | 82% | 85% | +3.3 | 11 | 423 |
| 38 | 39 | 1770 08 20 | 87% | 89% | 64% | 69% | 92% | 97% | 86% | 86% | 94% | 95% | 85% | 87% | +2.7 | 15 | 1,706 |
| 39 | 7 | 1768 05 02 | 90% | 75% | 78% | 82% | 77% | 86% | 95% | 96% | 95% | 93% | 87% | 86% | −0.8 | 10 | 805 |