Give to AI Formulas from Charts

Finds charts in PDF files and recovers the analytical formula of every curve on them.

No neural networks anywhere: the whole pipeline is deterministic, reproducible and explainable — every number in the output can be traced back to a specific geometric feature of the page.

$ analyze_pdf paper.pdf

Page 1 — source: vector PDF graphics
  Chart detected, confidence 0.96.
  X axis: "X", linear scale, range 0…10, 6 ticks, calibration R² 1.0000
  Y axis: "Y", linear scale, range 0…50, 6 ticks, calibration R² 1.0000
  Series 1 "linear A" (line, blue, 200 points), X ∈ [0; 10], Y ∈ [0.9868; 20.99]
     FORMULA: y = 2·x + 0.9868
     model "linear", R² = 1.00000, RMSE = 5.774e-13, 2 params
  Series 2 "quad B" (line, red, 200 points), X ∈ [0; 10], Y ∈ [-0.01318; 49.99]
     FORMULA: y = 0.5·x^2 + 0.0002635·x - 0.009103
     model "parabola", R² = 1.00000, RMSE = 0.002764, 3 params
  Series 3 "sine C" (line, green, 200 points), X ∈ [0; 10], Y ∈ [16.99; 32.98]
     FORMULA: y = 7.993·sin(0.8·x - 0.0007325) + 24.99
     model "sine", R² = 1.00000, RMSE = 0.007426, 4 params

Text extracted from the PDF (axis titles, curve labels) is of course reproduced in whatever language the document uses.

1. Decides whether the page contains a chart at all — weighted score over: two long perpendicular lines, short tick strokes touching them, numeric labels along the axes that fall on a straight line under regression, grid lines, and a polyline with many nodes inside the axes box. The decisive feature is thelinearity of the labels : for random text the regression R² is low, for a real axis it is ≈ 1.
2. Calibrates the axes — pixel → value regression with iterative worst-point rejection. A logarithmic-scale hypothesis is tested separately (same regression overlog10(value) ); this matters more than it sounds, because a straight line on a semi-log axisis an exponential, and without detecting the scale the formula comes out meaningless.
3. Extracts every curve and converts it to data coordinates.
4. Fits a formula — 11 models, winner picked by parsimony/AICc rather than by max R².

Two independent front-ends feed step 3, chosen automatically:

• Vector (src/vector.cpp ) — the main path. In a PDF a chart is stored as paths and text, so curve coordinates are read out of the file exactly, with no computer vision. Accuracy: fractions of a percent.
• Raster (src/raster.cpp ) — for scans and embedded images. Axes are found by morphological opening with a long kernel, labels are read with Tesseract, the curve is isolated by saturation/hue (for black curves: dark pixels minus long straight lines, i.e. minus grid and frame), then a per-column median gives the trace. Measured accuracy on the test scan: ≈ 0.3 % of the range.

When several curves share a chart, each series gets its own label:

• legend — if a short coloured swatch sits immediately left of a text run, the label is assigned to the seriesof that colour , not to the geometrically nearest curve (a legend usually sits in a corner, so "nearest curve" would hand every entry to whichever curve happens to pass by it);
• label next to the curve — otherwise the nearest series is taken, within 15 % of the shorter side of the plot box.

Matching is one-to-one and greedy by increasing cost. Text runs already consumed as axis numbers, axis titles or the chart title are excluded from the candidates. Vector branch only — see Limitations.

11 models: polynomials of degree 1–5, exponential, power, logarithm, sine, logistic, Gaussian, hyperbola, square root. Each gets a meaningful initial guess (log-linearisation for exponential and power, FFT peak plus mean-level crossing count for the sine, half-maximum position for the logistic) — with p0 = {1,1,1} almost nothing converges.

The winner is not the maximum R². By R² a high-degree polynomial always wins, because it eats the noise and the discretisation error. The rules, in order:

1. if several models reach R² ≥ 0.9999 — the one with fewer parameters wins;
2. otherwise, among models whose RSS is no worse than 1.6× the best — again fewest parameters;
3. inside that group — by AICc.

On synthetic data (11 dependency types × 2 noise levels) this rule scores 22/22.

Dependencies (Ubuntu 24.04):

apt-get install cmake ninja-build pkg-config \
  libmupdf-dev mupdf-tools libeigen3-dev libceres-dev \
  libgflags-dev libgoogle-glog-dev \
  libfreetype-dev libjpeg-dev libjbig2dec0-dev libopenjp2-7-dev \
  libharfbuzz-dev libgumbo-dev libmujs-dev \
  libopencv-dev libtesseract-dev tesseract-ocr tesseract-ocr-rus

tesseract-ocr-rus is only needed to read Cyrillic axis titles; everything else works without it.

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

Produces build/analyze_pdf.

./build/analyze_pdf chart.pdf                 # human-readable report
./build/analyze_pdf chart.pdf --json          # machine-readable
./build/analyze_pdf scan.pdf --csv out/       # also dump curve points as CSV
./build/analyze_pdf chart.pdf --force raster  # force the CV path
./build/analyze_pdf chart.pdf --force vector  # force the vector path
./build/analyze_pdf scan.pdf --dpi 300        # render resolution for the raster path

Tesseract prints its own diagnostics to stderr; stdout stays clean, so --json can be piped directly into a parser.

pdf_backend.hpp and raster.hpp are the only places that know about MuPDF and OpenCV/Tesseract respectively; the rest of the code works with their plain structs (PageContent, RawPath, TextSpan, PdfDocument::Raster).

reference/ holds the fixture PDFs plus two recorded outputs:

• expected_cpp.txt — what this implementation prints on all nine fixtures. Regenerate and diff it to catch regressions.
• expected.txt — the original Python implementation's output, in Russian. Kept for provenance; useful for comparingnumbers , not text.

for f in exp sin logy scatter_parabola power_en no_chart raster_exp multi_text multi_legend; do
  echo "########## $f.pdf"; ./build/analyze_pdf reference/$f.pdf 2>/dev/null; echo
done > /tmp/out.txt
diff /tmp/out.txt reference/expected_cpp.txt && echo "no regressions"

• Curve labels are vector-only. The raster path does not look for them yet — that needs OCR over the whole plot area rather than the narrow strips next to the axes, and it would keep catching the grid and the curves themselves.
• Same-coloured overlapping curves are not separated — they merge into one series.
• Closed and parametric curves (circle, hysteresis loop) are detected, but ay(x) formula is meaningless for them; the report flags the X-ambiguity.
• Bar and pie charts are recognised as "a chart", but the dependency model does not apply to them.
• Complex functions outside the 11-model library (sums of harmonics, damped oscillation, piecewise definitions) are not recognised as such — the tool still reports the best of the 11, just with a lower R². There is no explicit "I don't know this shape" signal beyond that R².
• Extrapolation past the plotted range is unreliable — the model was only fitted inside the visible window.
• Cyrillic in labels. matplotlib writes PDFs with Type3 fonts that carry no ToUnicode map, so the text layer returns garbage for Cyrillic. Handled by re-reading the title with OCR off a page render, which needstesseract-ocr-rus .
• Lost minus sign. The same Type3 fonts often drop the minus glyph, so an axis−4 −2 0 2 4 extracts as4 2 0 2 4 . Handled by testing "first/last k labels are negative" hypotheses and keeping the best R².

Both cost real debugging time and are not obvious from the MuPDF docs.

1. Do not flip the page coordinates yourself.fz_bound_page /fz_run_page already hand you a page space whose origin is top-left with y growing downwards — unlike the raw coordinates insidefz_path , whichfz_path_walker seesbefore thectm is applied. The transform you pass should therefore be a pure shift,fz_make_matrix(1,0,0,1,-x0,-y0) ; adding a flip mirrors the whole page.
2. Merge fill_path + stroke_path for the same path. The PDF operatorB (filland stroke) reaches anfz_device as two separate callbacks with the samefz_path* and the samectm . PyMuPDF'sget_drawings() reports this as a single object (type: "fs" , withfill andcolor together). Without merging them, the axes frame is counted twice and the grid-line/tick statistics in the detection score come out inflated.

• Solver — Ceres instead ofscipy.optimize.curve_fit . The winning model's formula and R² match the reference byte-for-byte almost everywhere; 2nd/3rd place in the "alternatives" list occasionally differs, because on deliberately bad models (a Gaussian fitted over a sine) Ceres converges to a different local optimum than scipy's LM. This never changed the winner in testing.
• --force vector really means vector-only. In the original,force was only branched on for"raster" ;"vector" did not disable the automatic raster fallback, which contradicted its own CLI help.
• Per-curve labels — new, the original identified series only by index and colour.
• The report is in English (the original printed Russian). The translation was verified by hashing every numeric token in the output before and after: identical, so only wording changed. As a side effect the report can no longer be byte-compared against the Python reference — hence the separatereference/expected_cpp.txt baseline.

AGPL-3.0-or-later — see LICENSE.

This is dictated by the dependency on MuPDF, which is AGPL (or a paid commercial licence from Artifex). Everything else here — Eigen (MPL2), Ceres (BSD), OpenCV (Apache-2.0), Tesseract (Apache-2.0) — is compatible with a more permissive licence. If you need one, replace the MuPDF backend with PDFium (BSD): the PDF-specific code is confined to src/pdf_backend.cpp behind the interface in include/plotparse/pdf_backend.hpp.