📄 Reading the unreadable: Creating a dataset of 19th century English newspapers using image-to-text language models

Summary

Digitised 19th-century newspapers often have missing or badly garbled OCR. This makes large archives hard to search or analyse. Bourne re-digitises the Nineteenth Century Serials Edition (NCSE), about 84k pages from six British periodicals. The pipeline is modular: it first detects page layout and bounding boxes, then sends cropped regions to Pixtral 12B, a general image-to-text language model, for transcription. A classification step then labels each entry by text type and by IPTC topic.

On two benchmarks, BLN600 and a newly hand-annotated NCSE test set, Pixtral reaches a median character error rate of about 1%. That is roughly five times better than the runner-up, Tesseract, and the processing cost is only a few dollars per thousand pages. The output is NCSE v2.0, an openly licensed corpus of 1.4 million entries and 321 million words. The paper shows example uses: comparing topics across periodicals, measuring readability and tracking events.

The study also finds that accurate region detection matters more than correcting OCR text afterwards. This makes it a useful guide for cheap, reproducible digitisation of historical archives.

Key points

• The source collection has 4,263 PDFs with 84,509 page images, covering six periodicals from 1806 to 1890 (for example the Northern Star, the Leader and the English Woman's Journal).
• Pixtral 12B was compared with four other OCR approaches, including Tesseract and GOT. Its CER was about 0.01 on both BLN600 and the NCSE test set, and the full run cost roughly $3.7 per 1,000 pages.
• The NCSE test set has 31 random pages with boxes drawn by hand in Labelbox. Each box was transcribed with line alignment, giving 358 image–text pairs.
• Pages were converted to 2-bit PNG at 120 dpi (200 dpi for the much larger Northern Star). This shrank the data from 60 GB to about 27 GB.
• NCSE v2.0 labels entries with 4 text types and 17 IPTC topics. It is released under CC BY on the UCL repository together with the test set, the bounding-box data and the silver sets, and the code is on GitHub.

Interesting ideas

• I1. Earlier LM post-correction of the old NCSE OCR only cut CER from 0.17 to 0.07, yet plain Tesseract reached 0.05 once regions were boxed correctly, so fixing segmentation upstream beats repairing its damage downstream.
• I2. Every pipeline stage has explicit inputs and outputs, so any component can be swapped out and scored on its own instead of judging only the final result.
• I3. An LLM labelled a modest silver set, and a much cheaper classifier trained on that set then labelled the whole corpus. The author suggests data-centric cleaning of the silver set as the next improvement.
• I4. Small vision–language models sometimes fell into repetition loops, and cropping the input into smaller regions greatly reduced this failure.
• I5. The author proposes rendering known text into synthetic newspaper layouts and adding noise, which gives fully controlled training and evaluation data where ground truth is known.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): Measure how much parse quality is lost to sentence-splitting and tokenization mistakes before adding repair rules later in the pipeline, and invest first in segmenting raw text cleanly.
• I2 (learn): Keep a separate gold check for each step (tokens, tree, frames, world) so a regression can be traced to the stage that caused it rather than seen only as a lower answer score.
• I3 (learn): Use the top LLM teacher tier to label a small sample, distil it into a fast local model for the bulk of the data, and audit the silver labels for noise before training on them.
• I4 (learn): When querying LLM teachers, send short, bounded chunks and automatically flag outputs with repeated spans as likely hallucinations before they enter silver data.
• I5 (back): Generate sentences from known meaning graphs, perturb them (typos, odd punctuation), and check whether the parse recovers the original graph, giving gold-labelled robustness tests at no annotation cost.