Solving Math From a Photo: What AI Can Read

Point a phone at a math problem, get a worked solution. It is the most-used feature in the whole category, and it is far stranger under the hood than it looks. Text OCR is a solved problem in the boring sense; math OCR is not, because math is not a string of characters. It is a two-dimensional structure, and the structure carries the meaning.

This is what these systems are actually doing, where they break, and how to give them a picture they can read.

Math is not text

Ordinary OCR reads left to right and returns a line of characters. That model fails immediately on a fraction, where the numerator sits above a bar and the denominator below it, and a symbol's position relative to its neighbours determines whether it is an exponent, an index, a limit bound, or a matrix entry.

So math recognition has always had two jobs stacked on top of each other. First, identify the symbols. Second, recover the relations between them: to the right of, above, below, superscript, subscript, inside. Classical systems did these as separate stages, and the second stage was a grammar parse over the recognised symbols. The failure mode was brutal — one misplaced relation reorganises the whole expression, so an otherwise perfect symbol recognition still returns nonsense.

What modern systems actually do

Almost everything shipping today is an image-to-markup model instead. A convolutional or transformer vision encoder maps the cropped image into a sequence of visual features; an autoregressive decoder emits markup — nearly always LaTeX — one token at a time, attending back to different regions of the image as it goes. These are trained on very large corpora of rendered LaTeX paired with its source, plus handwriting datasets from the long-running CROHME competition on handwritten expression recognition.

This design explains almost every behaviour users find puzzling.

It never returns garbage. The decoder is a language model over notation. It has learned that a backslash-frac takes two groups, that brackets close, that an integral usually acquires a differential. Faced with a smudge, it does not output uncertainty; it outputs the most probable well-formed expression consistent with what it can see. A misread is therefore a clean, plausible, entirely wrong equation.

Handwriting is harder than print, and offline handwriting is hardest. When recognition happens on a stylus device, the system sees stroke order, direction and timing — a rich signal. A photograph throws all of that away and leaves only pixels, which is the "offline" case, and accuracy drops accordingly.

Long expressions degrade faster than short ones. Errors are autoregressive: a wrong token changes the context for every token after it.

The symbols that break it

A short and depressingly stable list of ambiguities:

• 1, l, I and | — a lowercase L in a handwritten variable, the digit one, and an absolute-value bar are close to indistinguishable in many hands.
• x versus the multiplication cross versus the Greek chi. In algebra the cross is rare, so models are biased toward the variable — which is exactly wrong on an arithmetic worksheet.
• The horizontal stroke. Minus sign, fraction bar, overline (as in complex conjugate or repeating decimal), and the bar of an equals sign whose partner is faint all look alike. This is the highest-cost single error class, because a minus read as a fraction bar changes the shape of the whole problem.
• Superscript versus baseline drift. Handwriting rises and falls. "x2" as intended-squared and "x2" as sloppy multiplication are separated by a few pixels of vertical offset.
• Decimal points and separators. A comma decimal separator, standard across much of Europe, is routinely read as a list separator.
• 0, O, o and theta; 5 and S; 2 and z; 9, g and q. The usual suspects, worsened by the fact that in math all of them are plausible in context.
• Function application versus multiplication. f(x+1) can mean a function evaluated at x+1 or f times the quantity x+1, and no amount of image quality resolves it.
• The differential. In an integral, dx is a differential; elsewhere it may be d times x. Models get this right by convention, not by seeing it.

Diagrams are a different, unsolved problem

Reading the labels around a triangle works. Reading the triangle mostly does not. The information that a geometry problem actually depends on is often carried by drawing conventions rather than text: tick marks for congruent segments, a small square for a right angle, arrowheads for parallel lines, an arc for a marked angle, dashes for a hidden edge. Extraction of those is unreliable in every system we have tested, and figures are frequently not drawn to scale, so a model that infers relationships from apparent proportions infers wrong ones.

The practical consequence: if your answer depends on something only the picture tells you, type it. "Triangle ABC, angle C is a right angle, AB = 13, BC = 5, find AC" gets a correct result from a triangle solver every time; a photo of the same figure sometimes does not.

Graphs, circuit diagrams, free-body diagrams and number lines are harder still.

Word problems invert the difficulty

Interestingly, a photographed word problem is easy to read — it is ordinary printed prose — and hard to solve, because the difficulty has moved to translating English into equations. That step is done by the language model, not the vision system, so photo quality barely matters and phrasing matters a lot.

How to shoot a problem so it gets solved

1. One problem per photo. Crop tightly. Multi-problem worksheets force a layout-analysis step that adds a failure mode for no benefit.
2. Camera parallel to the page. Shooting at an angle produces keystone distortion that shifts baselines, which is precisely the signal used to detect exponents.
3. Even, indirect light. Direct overhead light on glossy textbook paper produces glare that erases strokes; the most common single problem is the photographer's own shadow across the page.
4. Dark pen, plain paper. Pencil on grid paper is the worst realistic case: low contrast plus printed lines the model may read as fraction bars or minus signs.
5. Exaggerate structure. Write exponents visibly smaller and higher. Make fraction bars clearly longer than any minus sign. Close every bracket.
6. Read the transcription. Any tool worth using shows you what it thinks the problem is before it solves it. That line is the only place a misread is visible — once the solution starts, everything below it is internally consistent and confidently wrong.

If the transcription is wrong, do not re-photograph in hope. Type the expression instead and hand it to a solver directly, whether that is an equation solver or an integral calculator. Thirty seconds of typing beats four attempts at a better angle.

Related reading:

• How AI actually solves math problems
• AI math accuracy: what the benchmarks mean