📄 Towards a Bulgarian Historical Newspaper Corpus – Construction of Reading Order over the Text in Searchable PDFs

Summary

Before any linguistic processing, a corpus of Bulgarian newspapers from 1878 to the mid-20th century needs the text on each page placed in its intended reading sequence. The sources are searchable PDFs from the "Ivan Vazov" National Library in Plovdiv. They were OCR-ed with ABBYY FineReader, which the team further trained on historical Cyrillic letters and non-standard fonts. Page layouts change constantly, so the text blocks in these PDFs have no dependable order, and articles end up interleaved. The authors built a page-processing algorithm that organizes extracted blocks into subpages and columns and then orders them. They compared it against plain off-the-shelf library extraction. For evaluation, they created a manually corrected gold dataset and scored outputs with a Levenshtein-type edit distance. The same metric guided a grid search over the algorithm's parameters, cutting the required edits by about 10%. A small editing tool supports the remaining manual fixes. The approach does not depend on the language and is a precondition for reliable article extraction at scale.

Key points

• The library's archival masters are 24-bit colour scans at 300 ppi. The public PDFs are lower-quality files produced by ABBYY FineReader, which was given targeted training snippets for obsolete letters such as yat and big nosovka.
• Over the gold dataset, the edit distance fell from 1925 with intuitive initial parameters to 1702 after a grid search of more than 20,000 parameter combinations.
• The tuned parameters include x_tolerance, partial_gap_threshold, min_column_page_ratio and min_column_width. Tuning per newspaper subset gave gains of up to 60% over the globally optimal settings.
• The output is XML structured as pages → subpages → columns → blocks → text. Manual post-editing relies on three operations: classifying blocks as normal, meta or noise; adjusting block coordinates; and swapping block order.
• The full collection exceeds 150,000 pages across more than 300 titles. Planned work includes fine-tuning an encoder transformer on subwords with 2D positions, plus article segmentation and spelling normalization. The work is supported by CLaDA-BG (CLARIN/DARIAH).