22 questions asked about each page image. Click column headers to sort.
NB: we categorize the question by what kind of task we foresee a human completing to answer the question rather than what that might be for LLMs, because the black box nature of LLMs prevents knowledge of which processes might be necessary for/or/employed by an LLM to answer the questions
| Q#▲ | Category by Human Task▲ | Question▲ | Answer Type▲ | Answer Format▲ |
|---|---|---|---|---|
| 1 | Feature identification | Is there a date (with year) near or at the top of the page? | Boolean (Y/N) | Y, N, or illegible |
| 2 | Transcription + Rule-based resolution | If so, what is the year? | Integer | 4-digit integer (e.g. 1768); leave blank if Q1 is N |
| 3 | Feature identification | Does the page have two columns? | Boolean (Y/N) | Y, N, or illegible |
| 4 | Feature identification | Is the page signed? | Boolean (Y/N) | Y, N, or illegible |
| 5 | Spatial location | If it is signed, where does the signature appear on the page? | String | Two words (e.g. bottom right or top center); leave blank if Q4 is N |
| 6 | Feature identification | Is there a small circular mark in the top left corner of the page? | Boolean (Y/N) | Y, N, or illegible |
| 7 | Counting | How many additional small circular marks are on the page (not part of a word, but as an annotation mark)? | Integer | Integer ≥ 0; excludes the top-left mark counted in Q6 |
| 8 | Counting | How many instances of quondam? | Integer | Integer ≥ 0 |
| 9 | Counting | How many instances of "con" as a preposition are on the page? | Integer | Integer ≥ 0 |
| 10 | Transcription | What two words immediately follow each instance of "con"? | Array of Strings | Pipe-separated list, one entry per instance: word1 word2 | word3 word4; leave blank if Q9 is 0 |
| 11 | Counting | How many instances of "e" as a conjunction? | Integer | Integer ≥ 0; include both e and ed |
| 12 | Transcription | What two words immediately follow "e" as a conjunction? | Array of Strings | Pipe-separated list, one entry per instance: word1 word2 | word3 word4; leave blank if Q11 is 0 |
| 13 | Counting | How many travelers are mentioned by name? | Integer | Integer ≥ 0 |
| 14 | Counting | How many travelers are mentioned who are not named? | Integer | Integer ≥ 0 |
| 15 | Counting | How many instances of place of origin and nationality describing the travelers appear in the document? | Integer | Integer ≥ 0; count each unique place and/or nationality once |
| 16 | Counting | How many unique places of stay are mentioned on the page? | Integer | Integer ≥ 0; count each unique place once across both columns |
| 17 | Counting + Spatial location | How many places of stay (count all) are mentioned per column? | Integer | Two integers separated by | in left-to-right order |
| 18 | Counting | How many indications of place of stay (i.e. 'in casa di [x]) include a person's name? | Integer | Integer ≥ 0; count every instance, even if the same name repeats |
| 19 | Counting | How many curly brackets appear on the page? | Integer | Integer ≥ 0 |
| 20 | Counting + Spatial location | How many curly brackets appear per column? | Integer | Two integers separated by | in left-to-right order |
| 21 | Feature identification | Where there are curly brackets, is there an annotation describing the bracketed information? | Boolean (Y/N) | Y, N, or illegible; leave blank if Q19 is 0 |
| 22 | Transcription | Write the words in the annotation. | String | Pipe-separated list, one entry per bracket: text1 | text2; leave blank if Q21 is N |