{"slug": "give-to-ai-formulas-from-charts", "title": "Give to AI Formulas from Charts", "summary": "A deterministic, non-neural-network pipeline extracts analytical formulas from charts embedded in PDF files, reading vector graphics directly and fitting one of 11 models to each curve. The tool reports R² = 1.00000 for a linear series (y = 2·x + 0.9868), a parabola (y = 0.5·x^2 + 0.0002635·x - 0.009103), and a sine (y = 7.993·sin(0.8·x - 0.0007325) + 24.99), with a raster fallback for scans and embedded images measured at approximately 0.3% of the range. The system selects the winning model by parsimony/AICc rather than maximum R², and detects logarithmic axes separately because a straight line on a semi-log axis is an exponential.", "body_md": "Finds charts in PDF files and recovers the analytical formula of every curve on them.\n\nNo neural networks anywhere: the whole pipeline is deterministic, reproducible and explainable — every number in the output can be traced back to a specific geometric feature of the page.\n\n``` bash\n$ analyze_pdf paper.pdf\n\nPage 1 — source: vector PDF graphics\n  Chart detected, confidence 0.96.\n  X axis: \"X\", linear scale, range 0…10, 6 ticks, calibration R² 1.0000\n  Y axis: \"Y\", linear scale, range 0…50, 6 ticks, calibration R² 1.0000\n  Series 1 \"linear A\" (line, blue, 200 points), X ∈ [0; 10], Y ∈ [0.9868; 20.99]\n     FORMULA: y = 2·x + 0.9868\n     model \"linear\", R² = 1.00000, RMSE = 5.774e-13, 2 params\n  Series 2 \"quad B\" (line, red, 200 points), X ∈ [0; 10], Y ∈ [-0.01318; 49.99]\n     FORMULA: y = 0.5·x^2 + 0.0002635·x - 0.009103\n     model \"parabola\", R² = 1.00000, RMSE = 0.002764, 3 params\n  Series 3 \"sine C\" (line, green, 200 points), X ∈ [0; 10], Y ∈ [16.99; 32.98]\n     FORMULA: y = 7.993·sin(0.8·x - 0.0007325) + 24.99\n     model \"sine\", R² = 1.00000, RMSE = 0.007426, 4 params\n```\n\nText extracted *from the PDF* (axis titles, curve labels) is of course reproduced in\nwhatever language the document uses.\n\n1. **Decides whether the page contains a chart at all** — weighted score over: two long\nperpendicular lines, short tick strokes touching them, numeric labels along the axes that\nfall on a straight line under regression, grid lines, and a polyline with many nodes\ninside the axes box. The decisive feature is the*linearity of the labels* : for random\ntext the regression R² is low, for a real axis it is ≈ 1.\n2. **Calibrates the axes** — pixel → value regression with iterative worst-point rejection.\nA logarithmic-scale hypothesis is tested separately (same regression over`log10(value)` );\nthis matters more than it sounds, because a straight line on a semi-log axis*is* an\nexponential, and without detecting the scale the formula comes out meaningless.\n3. **Extracts every curve** and converts it to data coordinates.\n4. **Fits a formula** — 11 models, winner picked by parsimony/AICc rather than by max R².\n\nTwo independent front-ends feed step 3, chosen automatically:\n\n- **Vector** (`src/vector.cpp` ) — the main path. In a PDF a chart is stored as paths and\ntext, so curve coordinates are read out of the file exactly, with no computer vision.\nAccuracy: fractions of a percent.\n- **Raster** (`src/raster.cpp` ) — for scans and embedded images. Axes are found by\nmorphological opening with a long kernel, labels are read with Tesseract, the curve is\nisolated by saturation/hue (for black curves: dark pixels minus long straight lines,\ni.e. minus grid and frame), then a per-column median gives the trace. Measured accuracy\non the test scan: ≈ 0.3 % of the range.\n\nWhen several curves share a chart, each series gets its own label:\n\n- **legend** — if a short coloured swatch sits immediately left of a text run, the label is\nassigned to the series*of that colour* , not to the geometrically nearest curve (a legend\nusually sits in a corner, so \"nearest curve\" would hand every entry to whichever curve\nhappens to pass by it);\n- **label next to the curve** — otherwise the nearest series is taken, within 15 % of the\nshorter side of the plot box.\n\nMatching is one-to-one and greedy by increasing cost. Text runs already consumed as axis\nnumbers, axis titles or the chart title are excluded from the candidates. Vector branch\nonly — see *Limitations*.\n\n11 models: polynomials of degree 1–5, exponential, power, logarithm, sine, logistic,\nGaussian, hyperbola, square root. Each gets a meaningful initial guess (log-linearisation\nfor exponential and power, FFT peak plus mean-level crossing count for the sine, half-maximum\nposition for the logistic) — with `p0 = {1,1,1}` almost nothing converges.\n\nThe winner is **not** the maximum R². By R² a high-degree polynomial always wins, because it\neats the noise and the discretisation error. The rules, in order:\n\n1. if several models reach R² ≥ 0.9999 — the one with fewer parameters wins;\n2. otherwise, among models whose RSS is no worse than 1.6× the best — again fewest parameters;\n3. inside that group — by AICc.\n\nOn synthetic data (11 dependency types × 2 noise levels) this rule scores 22/22.\n\nDependencies (Ubuntu 24.04):\n\n```\napt-get install cmake ninja-build pkg-config \\\n  libmupdf-dev mupdf-tools libeigen3-dev libceres-dev \\\n  libgflags-dev libgoogle-glog-dev \\\n  libfreetype-dev libjpeg-dev libjbig2dec0-dev libopenjp2-7-dev \\\n  libharfbuzz-dev libgumbo-dev libmujs-dev \\\n  libopencv-dev libtesseract-dev tesseract-ocr tesseract-ocr-rus\n```\n\n`tesseract-ocr-rus` is only needed to read Cyrillic axis titles; everything else works\nwithout it.\n\n```\ncmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release\ncmake --build build -j$(nproc)\n```\n\nProduces `build/analyze_pdf`.\n\n```\n./build/analyze_pdf chart.pdf                 # human-readable report\n./build/analyze_pdf chart.pdf --json          # machine-readable\n./build/analyze_pdf scan.pdf --csv out/       # also dump curve points as CSV\n./build/analyze_pdf chart.pdf --force raster  # force the CV path\n./build/analyze_pdf chart.pdf --force vector  # force the vector path\n./build/analyze_pdf scan.pdf --dpi 300        # render resolution for the raster path\n```\n\nTesseract prints its own diagnostics to stderr; stdout stays clean, so `--json` can be piped\ndirectly into a parser.\n\n| Path | Role | Libraries | \n|---|---|---|\n| `src/calib.cpp` | number parsing, axis calibration, log scale, minus-sign recovery | Eigen | \n| `src/pdf_backend.cpp` | MuPDF wrapper: paths, text runs, page rendering | MuPDF | \n| `src/vector.cpp` | axes, ticks, series, labels, \"is this a chart\" score | — | \n| `src/raster.cpp` | CV + OCR path | OpenCV, Tesseract | \n| `src/fit.cpp` | model library, initial guesses, parsimony/AICc selection | Eigen, Ceres | \n| `src/report.cpp` | report text, vector→raster fallback orchestration | — | \n| `src/main.cpp` | CLI | — | \n| `python-reference/` | the original Python implementation this was ported from (docs in Russian) | — | \n\n`pdf_backend.hpp` and `raster.hpp` are the only places that know about MuPDF and\nOpenCV/Tesseract respectively; the rest of the code works with their plain structs\n(`PageContent`, `RawPath`, `TextSpan`, `PdfDocument::Raster`).\n\n`reference/` holds the fixture PDFs plus two recorded outputs:\n\n- `expected_cpp.txt` — what this implementation prints on all nine fixtures. Regenerate and\ndiff it to catch regressions.\n- `expected.txt` — the original Python implementation's output, in Russian. Kept for\nprovenance; useful for comparing*numbers* , not text.\n\n```\nfor f in exp sin logy scatter_parabola power_en no_chart raster_exp multi_text multi_legend; do\n  echo \"########## $f.pdf\"; ./build/analyze_pdf reference/$f.pdf 2>/dev/null; echo\ndone > /tmp/out.txt\ndiff /tmp/out.txt reference/expected_cpp.txt && echo \"no regressions\"\n```\n\n| File | Ground truth | Expected result | \n|---|---|---|\n| `exp.pdf` | `y = 2e^{0.5x} − 1` | exponential, R² = 1.0 | \n| `sin.pdf` | `y = 4sin(1.3x + 0.4) + 2` | sine, R² ≈ 1.0 | \n| `logy.pdf` | `y = 5e^{0.8x}` , log Y axis | **Y axis = log** , exponential | \n| `scatter_parabola.pdf` | `y = 3x² − 2x + 7` , noise σ=2 | parabola, R² ≈ 0.994 | \n| `power_en.pdf` | `y = 1.7x^{2.3}` | power (NOT a polynomial) | \n| `raster_exp.pdf` | same as exp, but rasterised | raster path, R² ≈ 0.99999 | \n| `no_chart.pdf` | text and a table | no chart detected, score 0 | \n| `multi_text.pdf` | 3 curves labelled `A` /`B` /`C` next to each | 3 series, labels attached | \n| `multi_legend.pdf` | same 3 curves, labelled by legend | 3 series, labels via swatch colour | \n\n- **Curve labels are vector-only.** The raster path does not look for them yet — that needs\nOCR over the whole plot area rather than the narrow strips next to the axes, and it would\nkeep catching the grid and the curves themselves.\n- **Same-coloured overlapping curves are not separated** — they merge into one series.\n- **Closed and parametric curves** (circle, hysteresis loop) are detected, but a`y(x)` formula is meaningless for them; the report flags the X-ambiguity.\n- **Bar and pie charts** are recognised as \"a chart\", but the dependency model does not\napply to them.\n- **Complex functions outside the 11-model library** (sums of harmonics, damped oscillation,\npiecewise definitions) are not recognised as such — the tool still reports the best of the\n11, just with a lower R². There is no explicit \"I don't know this shape\" signal beyond\nthat R².\n- **Extrapolation past the plotted range is unreliable** — the model was only fitted inside\nthe visible window.\n- **Cyrillic in labels.** matplotlib writes PDFs with Type3 fonts that carry no ToUnicode\nmap, so the text layer returns garbage for Cyrillic. Handled by re-reading the title with\nOCR off a page render, which needs`tesseract-ocr-rus` .\n- **Lost minus sign.** The same Type3 fonts often drop the minus glyph, so an axis`−4 −2 0 2 4` extracts as`4 2 0 2 4` . Handled by testing \"first/last k labels are\nnegative\" hypotheses and keeping the best R².\n\nBoth cost real debugging time and are not obvious from the MuPDF docs.\n\n1. **Do not flip the page coordinates yourself.**`fz_bound_page` /`fz_run_page` already\nhand you a page space whose origin is top-left with y growing downwards — unlike the raw\ncoordinates inside`fz_path` , which`fz_path_walker` sees*before* the`ctm` is applied.\nThe transform you pass should therefore be a pure shift,`fz_make_matrix(1,0,0,1,-x0,-y0)` ; adding a flip mirrors the whole page.\n2. **Merge `fill_path` + `stroke_path` for the same path.** The PDF operator`B` (fill*and* stroke) reaches an`fz_device` as two separate callbacks with the same`fz_path*` and the same`ctm` . PyMuPDF's`get_drawings()` reports this as a single object\n(`type: \"fs\"` , with`fill` and`color` together). Without merging them, the axes frame is\ncounted twice and the grid-line/tick statistics in the detection score come out inflated.\n\n- **Solver** — Ceres instead of`scipy.optimize.curve_fit` . The winning model's formula and\nR² match the reference byte-for-byte almost everywhere; 2nd/3rd place in the \"alternatives\"\nlist occasionally differs, because on deliberately bad models (a Gaussian fitted over a\nsine) Ceres converges to a different local optimum than scipy's LM. This never changed the\nwinner in testing.\n- **`--force vector` really means vector-only.** In the original,`force` was only branched\non for`\"raster\"` ;`\"vector\"` did not disable the automatic raster fallback, which\ncontradicted its own CLI help.\n- **Per-curve labels** — new, the original identified series only by index and colour.\n- **The report is in English** (the original printed Russian). The translation was verified\nby hashing every numeric token in the output before and after: identical, so only wording\nchanged. As a side effect the report can no longer be byte-compared against the Python\nreference — hence the separate`reference/expected_cpp.txt` baseline.\n\nAGPL-3.0-or-later — see [LICENSE](/BorisYamp/plotparse/blob/main/LICENSE).\n\nThis is dictated by the dependency on **MuPDF**, which is AGPL (or a paid commercial licence\nfrom Artifex). Everything else here — Eigen (MPL2), Ceres (BSD), OpenCV (Apache-2.0),\nTesseract (Apache-2.0) — is compatible with a more permissive licence. If you need one,\nreplace the MuPDF backend with PDFium (BSD): the PDF-specific code is confined to\n`src/pdf_backend.cpp` behind the interface in `include/plotparse/pdf_backend.hpp`.", "url": "https://wpnews.pro/news/give-to-ai-formulas-from-charts", "canonical_source": "https://github.com/BorisYamp/plotparse", "published_at": "2026-09-13 09:52:22+00:00", "updated_at": "2026-09-13 10:05:31.903464+00:00", "lang": "en", "topics": ["ai-tools", "computer-vision", "developer-tools", "ai-research"], "entities": ["Tesseract"], "alternates": {"html": "https://wpnews.pro/news/give-to-ai-formulas-from-charts", "markdown": "https://wpnews.pro/news/give-to-ai-formulas-from-charts.md", "text": "https://wpnews.pro/news/give-to-ai-formulas-from-charts.txt", "jsonld": "https://wpnews.pro/news/give-to-ai-formulas-from-charts.jsonld"}}