Image to Text

How optical character recognition actually works

OCR turns a picture of writing back into editable characters, and it does so in stages. First the image is cleaned up and converted to high-contrast black and white, and the engine works out which way is up and corrects any skew. Next it segments the page — finding blocks of text, then lines, then individual word and character shapes. Finally, each isolated shape is classified: a trained model compares it against what it knows letters look like and outputs its best guess, refined by a language model that knows 'tbe' is far more likely to be 'the'. Every one of those stages depends on the previous one, which is exactly why a blurred or crooked photo fails so badly — if segmentation cannot find clean character shapes, the smartest classifier in the world has nothing useful to work with.

Getting the most out of it

The single biggest lever on OCR accuracy is not the software, it is the input. A few habits make a dramatic difference. Shoot straight down at the page rather than from an angle, so the characters are not distorted by perspective. Get plenty of diffuse light on the page and avoid a hard glare or your own shadow falling across it. Fill the frame — the more pixels each letter occupies, the more the engine has to work with, and text smaller than about 20 pixels tall is often hopeless. Crop out irrelevant background before scanning. And always select the correct language first, because the language model does a great deal of the heavy lifting in resolving ambiguous characters.