What OCR Actually Does, and Why the Columns Come Back Interleaved

You ran a scan or a screenshot through a free OCR converter and got back a wall of text: two columns welded into one string, a table with no rows left in it, a "0" where the word clearly had an "O". That output isn't a bad run. It's what the software is built to produce, and understanding why explains both the mess and what a different approach fixes. The same problem shows up whether you're holding a photo or a scanned PDF; the PDF itself can go in directly instead of a screenshot of it, and everything below applies the same way.

OCR stands for optical character recognition, and the name is literal. It recognizes characters. It does three things, in order, and nothing more: it sweeps the image for shapes that look like letters, it matches each shape against a library of known letterforms, and it prints the matches out in the order it came across them on the page. It never reads a sentence. It never knows it's looking at an invoice, a whiteboard, or a menu. It sees marks, and marks either look like an "S" or they don't.

Why layout disappears first

A page has layout: a heading up top, two columns side by side, a total boxed off in the corner. That arrangement is information. It tells a reader which price goes with which item, and which line is a heading rather than a sentence.

A shape-matcher doesn't see any of that. It works across the image roughly left to right, top to bottom, and prints each letter as it's found. Two columns sitting next to each other get read a line at a time, alternating between them, so the left column's item names end up welded to the right column's prices. A total that was boxed off in the corner lands mid-sentence in whatever text happened to be nearby on the page. Nothing was lost. It just never had a way to keep two things separate that weren't already one continuous line of type.

Why the arrows and circles disappear

A whiteboard photo usually carries more than words: an arrow linking two boxes, a circled name, a star next to the item that matters. On a real board, that's part of what the writing means.

Shape-matching has no answer for any of it, because there's no character an arrow matches to. It isn't approximated or flattened into text. It's dropped, silently, and the reply gives no sign that anything was there.

Why a 5 becomes an S, and a 1 becomes an L

Handwriting is genuinely ambiguous at the level of a single mark. A hurried 5 and a capital S are often the same shape, a 1 and a lowercase L look nearly identical, and "rn" written close together can pass for a single "m". A person reading a note gets these right without noticing, because the rest of the sentence rules out the wrong answer: "that's clearly a price, so it's a 5."

A shape-matcher has no sentence to check against. It sees the mark alone and picks the closest letterform in its library. Sometimes that's right. On the marks that are genuinely ambiguous, it's a coin flip with no way to know which side it landed on. Run that coin flip a few hundred times down a page of handwriting and the result is the wall of near-nonsense you already got back. It isn't one bad guess. It's many small ones, none of them checked against anything.

So what do I use instead?

The alternative isn't a better version of the same sweep. It's a different question being asked of the picture. Instead of "what letter shapes are here", it's "give me the left column as a list", "what does this error mean", "which of these receipts is the largest", the same way you'd ask a person looking over your shoulder. In the image analyzer, the reply is written to answer that question, not printed in scan order.

Take the two-column receipt from above and ask "give me the left column as a list." What comes back is the item names on their own, one per line, no prices welded on, because the question was about the column, not about the order the shapes sat in. Ask instead for the arrow linking two boxes on a whiteboard, and it doesn't vanish: the reply says something like "an arrow points from 'Q3 budget' to 'hiring plan.'" And if you add "mark anything you're not sure about," a smudged total comes back as "$4?.00, unclear" instead of a clean number standing in for a guess.

Reading with context fixes exactly the three failures above, because context is what shape-matching never had. A layout question ("is this two columns or one") gets answered from what the columns visibly are, not from the order pixels were swept. A symbol gets described, not dropped, because nothing here is limited to a fixed letter library. And an ambiguous mark gets settled the way a person settles it: the surrounding words narrow it down. A smudged digit in the middle of "total: $4_.00" only has two or three sensible completions, and the sentence around it picks one. The exact wording to use for a whiteboard, a receipt pile, or a page of notes is covered picture by picture over here.

The cost of that fix: a plausible guess looks exactly like a fact

Context cuts both ways, and this is the one thing worth being honest about before you trust a result. On an ordinary smeared word inside a sentence, context is a correction: there's a right answer, and the surrounding words point to it. On an invoice number, a serial number, or a total, there usually isn't a sentence around the digits pulling toward one answer. Nothing about "INV-04582" tells you whether that middle digit is a 4 or a 9. If a character there is genuinely unreadable, a system built to give you a coherent answer can still return one, filled in from what looks statistically likely rather than what was actually printed. It comes back typeset the same as everything else, with nothing marking it as a guess.

That's the reason to check numbers, proper nouns, and anything you'd be embarrassed to get wrong, and to ask the reply to flag anything it isn't sure about instead of guessing. It's also why zooming into the original photo first is worth the five seconds: if you can't read a digit at full magnification, no amount of context invents the pixels that were never captured. Eight images go into a single pass at once, so a stack of twenty receipts is three passes, and each pass is one round of checking, not twenty.