Old OCR text cripples language model training, and FineBooks wants to fix that at scale
The FineBooks project from Hugging Face and EleutherAI tested 14 open-weight OCR models on more than 2,000 pages from historical books. The best models already produce text good enough for AI training, but they aren’t ready for scholarly use.
Training open-source AI language models on public-domain books means dealing with bad text. Libraries extracted those texts from scans years ago using optical character recognition, and the results are often full of errors. The Talkie project put a number on the potential damage: a language model trained on OCR text learned at only 30 percent the efficiency of one trained on human transcriptions of the same books. FineBooks, a collaboration between Hugging Face and EleutherAI, tested whether current open-source OCR models can solve this problem. The team ran 14 open-weights models on 2,165 historical book pages and published the results as a leaderboard. The best models hit character accuracy above 97 percent at less than two dollars per thousand pages.Ad Millions of public-domain pages need better text recognition When EleutherAI and its partners released Common Pile last year, the largest openly licensed training corpus to date, it contained about 300,000 public-domain books with text from older OCR runs. The FineBooks authors say reprocessing those books with better models is one of the most effective ways to improve open AI training datasets.AdDEC_D_Incontent-1 The project picked the Biodiversity Heritage Library (BHL) as its first target, which holds more than 300,000 digitized natural history documents totaling over 64 million pages. The BHL offers its collection as a bulk download via AWS. Measuring OCR quality requires pages with known correct transcriptions. The team used work from the IMPACT project and BHL-Europe: between 2011 and 2012, experts transcribed six BHL volumes in English, French, German, and Latin with an error rate of about one character per 2,000. That data is available under a CC-BY license in a GitHub repository and forms the basis of the new ground-truth dataset.Ad All 14 models are freely available and run on local hardware without an API key. The metric is Character Error Rate (CER), the share of incorrectly recognized characters. The leaderboard splits results into a “diplomatic” variant that counts modernization of archaic characters like the long s (ſ) as an error and a “reading” variant that tolerates such changes. The leading dots.mocr model uses just 3 billion parameters, while Qwen3.5-9B scores lower despite being nearly three times as large. OvisOCR2 takes second place with only 0.9 billion parameters at 46 cents per thousand pages.
