Image to text (OCR): how it works and how to get clean text

Updated:

OCR, optical character recognition, turns a picture of text into text you can copy, search and edit. It is what reads a screenshot of an error message, a photo of a letter or a scanned contract. Most of its mistakes come from the image, not the engine, so a few minutes spent on the picture usually save more time than correcting the result.

How OCR reads an image

Tesseract, the open-source engine behind most free OCR tools, works in three steps. It first turns the image into black and white, deciding for each pixel whether it is ink or paper. It then finds the blocks, columns and lines of text. Finally, since version 4.00, a neural network reads each whole line at once, rather than letter by letter, and its language data helps it choose between similar shapes.

Each step can go wrong in its own way: a shadow across the page spoils the black and white, a tilted photo spoils the lines, and the wrong language spoils the letters.

Image to text (OCR)

What makes the biggest difference

Common problems and what to do

What you seeWhat to do
Letters missing or brokenTake the photo in even light, without shadows; increase contrast or scan at 300 dpi
Lines merged or out of orderStraighten the image; for receipts and tables, read line by line
Gibberish at the edgesCrop off the scanner’s dark borders, leaving a narrow white margin
Accents lostChoose the language of the text before reading
Light text on a dark background read badlyInvert the colours first: the engine expects dark text on a light background
0 and O, 1 and l, 5 and S mixed upCheck codes, amounts and names by eye: there are no words to help the engine guess

Errors in numbers matter most, because nothing in the text gives them away. Always compare amounts, dates and reference numbers with the image.

Screenshots, photos and scans

A screenshot is the easy case: sharp, straight and evenly lit. Its only problem is small interface text, which the nTools tool enlarges before reading. A photo of a page brings the hard cases: perspective, shadows, blur and a busy background. A flatbed scan at 300 dpi is the best source for long documents.

Scanned PDFs

A PDF made from a scan is a set of pictures: you cannot select its text, a search finds nothing, and a screen reader has nothing to read. The W3C’s accessibility guidance (WCAG) treats such a PDF as images of text and recommends running OCR on it, then checking the recognised text and its reading order. If you can select the text in a PDF, it already is text, and copying it is more accurate than any OCR.

PDF to Word reads scanned pages with OCR and gives an editable document; you can also turn the pages into images and read them in Image to text.

PDF to Word

Where the image goes

Many OCR services upload the image to a server. Tesseract can also run inside the browser, compiled to WebAssembly: that is how the nTools tool works, so the image and the text stay on your device. The browser only downloads the engine and the data for the language you choose.

Frequently asked questions

Can OCR read handwriting?

Not well. Tesseract is trained on printed text; neat block capitals sometimes work, ordinary handwriting does not. Handwriting needs engines trained specifically for it.

How accurate is OCR?

On a clean printed page scanned at 300 dpi, nearly every character is right. Photos, small text and unusual fonts bring more errors. The confidence figure the tool shows is the engine’s own estimate, a warning sign rather than a guarantee.

Why are 0 and O, or 1 and l, mixed up?

Their shapes are almost the same in many fonts. In ordinary words the language data helps choose; in codes, serial numbers and passwords it cannot, so check those character by character.

Which languages can it read?

Tesseract has data for more than 100 languages. The nTools tool offers English, Spanish, French, German, Italian and Portuguese; choose the language before reading, because it changes which letters the engine expects.

Does OCR keep tables and formatting?

No. The result is plain text with line breaks. Tables are a known weakness of Tesseract: read them line by line, so each row stays together, and rebuild the columns afterwards.

References

  1. Tesseract documentation — Improving the quality of the output
  2. Tesseract documentation — Neural nets in Tesseract 4.00
  3. Tesseract.js — Pure JavaScript OCR for more than 100 languages
  4. W3C — WCAG 2.2 technique PDF7: performing OCR on a scanned PDF document
  5. W3C — Understanding WCAG 2.2: Images of Text (1.4.5)

Related guides