Give to AI Formulas from Charts

A deterministic, non-neural-network pipeline extracts analytical formulas from charts embedded in PDF files, reading vector graphics directly and fitting one of 11 models to each curve. The tool reports R² = 1.00000 for a linear series (y = 2·x + 0.9868), a parabola (y = 0.5·x^2 + 0.0002635·x - 0.009103), and a sine (y = 7.993·sin(0.8·x - 0.0007325) + 24.99), with a raster fallback for scans and embedded images measured at approximately 0.3% of the range. The system selects the winning model by parsimony/AICc rather than maximum R², and detects logarithmic axes separately because a straight line on a semi-log axis is an exponential.

Finds charts in PDF files and recovers the analytical formula of every curve on them. No neural networks anywhere: the whole pipeline is deterministic, reproducible and explainable — every number in the output can be traced back to a specific geometric feature of the page. bash $ analyze pdf paper.pdf Page 1 — source: vector PDF graphics Chart detected, confidence 0.96. X axis: "X", linear scale, range 0…10, 6 ticks, calibration R² 1.0000 Y axis: "Y", linear scale, range 0…50, 6 ticks, calibration R² 1.0000 Series 1 "linear A" line, blue, 200 points , X ∈ 0; 10 , Y ∈ 0.9868; 20.99 FORMULA: y = 2·x + 0.9868 model "linear", R² = 1.00000, RMSE = 5.774e-13, 2 params Series 2 "quad B" line, red, 200 points , X ∈ 0; 10 , Y ∈ -0.01318; 49.99 FORMULA: y = 0.5·x^2 + 0.0002635·x - 0.009103 model "parabola", R² = 1.00000, RMSE = 0.002764, 3 params Series 3 "sine C" line, green, 200 points , X ∈ 0; 10 , Y ∈ 16.99; 32.98 FORMULA: y = 7.993·sin 0.8·x - 0.0007325 + 24.99 model "sine", R² = 1.00000, RMSE = 0.007426, 4 params Text extracted from the PDF axis titles, curve labels is of course reproduced in whatever language the document uses. 1. Decides whether the page contains a chart at all — weighted score over: two long perpendicular lines, short tick strokes touching them, numeric labels along the axes that fall on a straight line under regression, grid lines, and a polyline with many nodes inside the axes box. The decisive feature is the linearity of the labels : for random text the regression R² is low, for a real axis it is ≈ 1. 2. Calibrates the axes — pixel → value regression with iterative worst-point rejection. A logarithmic-scale hypothesis is tested separately same regression over log10 value ; this matters more than it sounds, because a straight line on a semi-log axis is an exponential, and without detecting the scale the formula comes out meaningless. 3. Extracts every curve and converts it to data coordinates. 4. Fits a formula — 11 models, winner picked by parsimony/AICc rather than by max R². Two independent front-ends feed step 3, chosen automatically: - Vector src/vector.cpp — the main path. In a PDF a chart is stored as paths and text, so curve coordinates are read out of the file exactly, with no computer vision. Accuracy: fractions of a percent. - Raster src/raster.cpp — for scans and embedded images. Axes are found by morphological opening with a long kernel, labels are read with Tesseract, the curve is isolated by saturation/hue for black curves: dark pixels minus long straight lines, i.e. minus grid and frame , then a per-column median gives the trace. Measured accuracy on the test scan: ≈ 0.3 % of the range. When several curves share a chart, each series gets its own label: - legend — if a short coloured swatch sits immediately left of a text run, the label is assigned to the series of that colour , not to the geometrically nearest curve a legend usually sits in a corner, so "nearest curve" would hand every entry to whichever curve happens to pass by it ; - label next to the curve — otherwise the nearest series is taken, within 15 % of the shorter side of the plot box. Matching is one-to-one and greedy by increasing cost. Text runs already consumed as axis numbers, axis titles or the chart title are excluded from the candidates. Vector branch only — see Limitations . 11 models: polynomials of degree 1–5, exponential, power, logarithm, sine, logistic, Gaussian, hyperbola, square root. Each gets a meaningful initial guess log-linearisation for exponential and power, FFT peak plus mean-level crossing count for the sine, half-maximum position for the logistic — with p0 = {1,1,1} almost nothing converges. The winner is not the maximum R². By R² a high-degree polynomial always wins, because it eats the noise and the discretisation error. The rules, in order: 1. if several models reach R² ≥ 0.9999 — the one with fewer parameters wins; 2. otherwise, among models whose RSS is no worse than 1.6× the best — again fewest parameters; 3. inside that group — by AICc. On synthetic data 11 dependency types × 2 noise levels this rule scores 22/22. Dependencies Ubuntu 24.04 : apt-get install cmake ninja-build pkg-config \ libmupdf-dev mupdf-tools libeigen3-dev libceres-dev \ libgflags-dev libgoogle-glog-dev \ libfreetype-dev libjpeg-dev libjbig2dec0-dev libopenjp2-7-dev \ libharfbuzz-dev libgumbo-dev libmujs-dev \ libopencv-dev libtesseract-dev tesseract-ocr tesseract-ocr-rus tesseract-ocr-rus is only needed to read Cyrillic axis titles; everything else works without it. cmake -S . -B build -G Ninja -DCMAKE BUILD TYPE=Release cmake --build build -j$ nproc Produces build/analyze pdf . ./build/analyze pdf chart.pdf human-readable report ./build/analyze pdf chart.pdf --json machine-readable ./build/analyze pdf scan.pdf --csv out/ also dump curve points as CSV ./build/analyze pdf chart.pdf --force raster force the CV path ./build/analyze pdf chart.pdf --force vector force the vector path ./build/analyze pdf scan.pdf --dpi 300 render resolution for the raster path Tesseract prints its own diagnostics to stderr; stdout stays clean, so --json can be piped directly into a parser. | Path | Role | Libraries | |---|---|---| | src/calib.cpp | number parsing, axis calibration, log scale, minus-sign recovery | Eigen | | src/pdf backend.cpp | MuPDF wrapper: paths, text runs, page rendering | MuPDF | | src/vector.cpp | axes, ticks, series, labels, "is this a chart" score | — | | src/raster.cpp | CV + OCR path | OpenCV, Tesseract | | src/fit.cpp | model library, initial guesses, parsimony/AICc selection | Eigen, Ceres | | src/report.cpp | report text, vector→raster fallback orchestration | — | | src/main.cpp | CLI | — | | python-reference/ | the original Python implementation this was ported from docs in Russian | — | pdf backend.hpp and raster.hpp are the only places that know about MuPDF and OpenCV/Tesseract respectively; the rest of the code works with their plain structs PageContent , RawPath , TextSpan , PdfDocument::Raster . reference/ holds the fixture PDFs plus two recorded outputs: - expected cpp.txt — what this implementation prints on all nine fixtures. Regenerate and diff it to catch regressions. - expected.txt — the original Python implementation's output, in Russian. Kept for provenance; useful for comparing numbers , not text. for f in exp sin logy scatter parabola power en no chart raster exp multi text multi legend; do echo " $f.pdf"; ./build/analyze pdf reference/$f.pdf 2 /dev/null; echo done /tmp/out.txt diff /tmp/out.txt reference/expected cpp.txt && echo "no regressions" | File | Ground truth | Expected result | |---|---|---| | exp.pdf | y = 2e^{0.5x} − 1 | exponential, R² = 1.0 | | sin.pdf | y = 4sin 1.3x + 0.4 + 2 | sine, R² ≈ 1.0 | | logy.pdf | y = 5e^{0.8x} , log Y axis | Y axis = log , exponential | | scatter parabola.pdf | y = 3x² − 2x + 7 , noise σ=2 | parabola, R² ≈ 0.994 | | power en.pdf | y = 1.7x^{2.3} | power NOT a polynomial | | raster exp.pdf | same as exp, but rasterised | raster path, R² ≈ 0.99999 | | no chart.pdf | text and a table | no chart detected, score 0 | | multi text.pdf | 3 curves labelled A / B / C next to each | 3 series, labels attached | | multi legend.pdf | same 3 curves, labelled by legend | 3 series, labels via swatch colour | - Curve labels are vector-only. The raster path does not look for them yet — that needs OCR over the whole plot area rather than the narrow strips next to the axes, and it would keep catching the grid and the curves themselves. - Same-coloured overlapping curves are not separated — they merge into one series. - Closed and parametric curves circle, hysteresis loop are detected, but a y x formula is meaningless for them; the report flags the X-ambiguity. - Bar and pie charts are recognised as "a chart", but the dependency model does not apply to them. - Complex functions outside the 11-model library sums of harmonics, damped oscillation, piecewise definitions are not recognised as such — the tool still reports the best of the 11, just with a lower R². There is no explicit "I don't know this shape" signal beyond that R². - Extrapolation past the plotted range is unreliable — the model was only fitted inside the visible window. - Cyrillic in labels. matplotlib writes PDFs with Type3 fonts that carry no ToUnicode map, so the text layer returns garbage for Cyrillic. Handled by re-reading the title with OCR off a page render, which needs tesseract-ocr-rus . - Lost minus sign. The same Type3 fonts often drop the minus glyph, so an axis −4 −2 0 2 4 extracts as 4 2 0 2 4 . Handled by testing "first/last k labels are negative" hypotheses and keeping the best R². Both cost real debugging time and are not obvious from the MuPDF docs. 1. Do not flip the page coordinates yourself. fz bound page / fz run page already hand you a page space whose origin is top-left with y growing downwards — unlike the raw coordinates inside fz path , which fz path walker sees before the ctm is applied. The transform you pass should therefore be a pure shift, fz make matrix 1,0,0,1,-x0,-y0 ; adding a flip mirrors the whole page. 2. Merge fill path + stroke path for the same path. The PDF operator B fill and stroke reaches an fz device as two separate callbacks with the same fz path and the same ctm . PyMuPDF's get drawings reports this as a single object type: "fs" , with fill and color together . Without merging them, the axes frame is counted twice and the grid-line/tick statistics in the detection score come out inflated. - Solver — Ceres instead of scipy.optimize.curve fit . The winning model's formula and R² match the reference byte-for-byte almost everywhere; 2nd/3rd place in the "alternatives" list occasionally differs, because on deliberately bad models a Gaussian fitted over a sine Ceres converges to a different local optimum than scipy's LM. This never changed the winner in testing. - --force vector really means vector-only. In the original, force was only branched on for "raster" ; "vector" did not disable the automatic raster fallback, which contradicted its own CLI help. - Per-curve labels — new, the original identified series only by index and colour. - The report is in English the original printed Russian . The translation was verified by hashing every numeric token in the output before and after: identical, so only wording changed. As a side effect the report can no longer be byte-compared against the Python reference — hence the separate reference/expected cpp.txt baseline. AGPL-3.0-or-later — see LICENSE /BorisYamp/plotparse/blob/main/LICENSE . This is dictated by the dependency on MuPDF , which is AGPL or a paid commercial licence from Artifex . Everything else here — Eigen MPL2 , Ceres BSD , OpenCV Apache-2.0 , Tesseract Apache-2.0 — is compatible with a more permissive licence. If you need one, replace the MuPDF backend with PDFium BSD : the PDF-specific code is confined to src/pdf backend.cpp behind the interface in include/plotparse/pdf backend.hpp .