Show HN: Folate – Local DOM selector model that beats 7B VLMs at 1/60th the size

PyPI • Hugging Face • Demo • Apache 2.0

A lightweight, CPU-efficient neural DOM element ranker built for browser automation within tight memory constraints (<2 GB total footprint alongside a headless browser).

Fine-tuned from google/electra-base-discriminator on osunlp/Mind2Web and dynamically quantized to INT8 ONNX (~107 MB), Folate runs purely via onnxruntime and tokenizers—requiring no PyTorch or transformers at inference time.

Folate operates as a discriminative cross-encoder that scores (natural_language_query, serialized_dom_element) pairs in a single batched forward pass on CPU.

• Zero-Hallucination Selector Grounding — Ranks real interactive DOM nodes extracted vialxml XPath (<a> ,<button> ,<input> ,<select> ,<textarea> , ARIA interactive roles, andonclick /tabindex elements), guaranteeing every returned CSS/XPath selector exists on the live page.
• Compact Structured Serialization — Serializes each element into a concise96 -token representation:[TAG] ... [ID/NAME/TYPE/ROLE/ARIA-LABEL/PLACEHOLDER/TITLE/ALT/HREF/DATA-TESTID] ... [TEXT] ... [PARENT] ...
• Edge & Low-Memory Deployment — Designed to run alongside Chromium on resource-constrained laptops and containers without GPU acceleration.

pip install folate

Or install from source:

git clone https://github.com/figranium/folate
cd folate
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

To run the live browser examples with Playwright:

python -m pip install -r examples/requirements.txt
python -m playwright install chromium

FolateSelectorFinder automatically downloads and caches figranium/folate on first run (or loads ./folate-int8 if present locally):

from folate import FolateSelectorFinder

finder = FolateSelectorFinder()
matches = finder.rank("click Search Flights button", html_source, top_k=3)

for match in matches:
    print(f"{match['score']:.4f} | {match['selector']} | {match['element']}")

To load weights from a custom local directory:

finder = FolateSelectorFinder(model_dir="./folate-int8")

If you want to pre-fetch the quantized model weights via huggingface_hub:

hf download figranium/folate --local-dir ./folate-int8
folate/
├── folate/
│   ├── __init__.py
│   └── ranker.py                  # Lightweight CPU inference engine & DOM serializer
├── scripts/
│   ├── prepare_data.py            # Streaming Mind2Web serializer (lxml)
│   ├── train.py                   # Fine-tuning script (bf16/fp16/CPU)
│   ├── export_onnx.py             # FP32 ONNX export + INT8 dynamic quantization
│   ├── eval_mind2web.py           # Official Mind2Web evaluator (Precision/Recall/F1, Top-1/3/5 & MRR)
│   └── eval_precision.py          # Playwright DOM-resolved selector Precision & Recall benchmark
├── examples/
│   ├── test_google.py             # Live Playwright search box grounding on google.com
│   ├── test_amazon.py             # E-commerce element ranking on amazon.com (+ offline snapshot)
│   ├── test_github.py             # Repository navigation grounding on github.com
│   └── requirements.txt           # Optional Playwright dependency for live browser tests
├── requirements.txt               # Lightweight CPU runtime dependencies
├── requirements-train.txt         # Training & ONNX export dependencies
└── LICENSE

Folate is evaluated on the three standard generalization splits of Mind2Web (test_task, test_website, and test_domain) across 6,037 action steps on real-world webpages.

• Task Formulation: At each step, the model receives the natural-language task instruction, action history context, and up to50 candidate DOM elements (Avg Candidates ≈ 50.0 ) extracted from the page snapshot, and outputs a relevance logit per candidate.
• Metrics: - Top-1 / Top-3 / Top-5: Percentage of steps where a ground-truth target node (pos_candidates ) appears within the topk ranked elements.
  • Precision / Recall / F1: Selective Top-1 precision (TP / (TP + FP) ) and recall (TP / Total ) across confidence probability thresholds (--confidence-threshold ,--thresholds ), plus candidate-level binary precision/recall.
  • MRR (Mean Reciprocal Rank): Average reciprocal rank for the highest-ranked ground-truth element across all evaluated steps.
• Hardware & Runtime: Latency is measured end-to-end per step on2 vCPUs withno GPU acceleration , using the exported INT8 quantized ONNX artifact (onnxruntime ). Variance betweentest_website (4,328.2 ms/step ) andtest_domain (2,431.0 ms/step ) reflects longer tokenized DOM attribute sequences on unseen site templates rather than candidate count.

For context against a fast, cost-efficient frontier LLM, GPT-6 Luna (via OpenAI's Decisions API) scores 81.8% on Mind2Web Multimodal (Google Gemini Decision 1 evaluation, Nov 2026) and 81.2% on Mind2Web in the Decision Index Vision suite (Nace.AI, Nov 2026), with ~326 ms median cloud API latency and ~$0.10 per 1,000 decisions (or $0.10 / $0.50 per 1M input/output tokens in standard API usage).

Note: Folate is evaluated on text-based Mind2Web DOM candidates (50 candidates/step), whereas GPT-6 Luna's 81.2%–81.8% scores are on multimodal/screenshot-based Mind2Web action selection. While GPT-6 Luna is a general-purpose frontier model capable of broad reasoning and multi-step planning, Folate's task-specific ELECTRA cross-encoder achieves higher single-step DOM element grounding accuracy locally on CPU with zero API cost, zero network dependency, and zero selector hallucination.

Run any of the included live browser examples from the repository root:

python examples/test_google.py
python examples/test_amazon.py --offline
python examples/test_github.py

To reproduce data preparation, fine-tuning, INT8 ONNX export, and evaluation from scratch on osunlp/Mind2Web:

python -m pip install -r requirements-train.txt

python scripts/prepare_data.py --output-dir ./data

python scripts/train.py --data-dir ./data --save-dir ./folate-base

python scripts/export_onnx.py --model-dir ./folate-base --output-dir ./folate-int8

python scripts/eval_mind2web.py --confidence-threshold 0.5 --json-out mind2web_results.json

python scripts/eval_precision.py --confidence-threshold 0.5 --json-out precision_results.json

• Intended Use: Single-step natural-language UI element grounding (click ,type ,select ,hover ) in local browser automation pipelines where RAM and CPU budgets are constrained.
• Limitations: - No Multi-Step Planning: Folate ranks candidate elements for a single action query; it does not plan multi-step workflows or parse unstructured page content.
  • Local DOM Context Only: Elements are serialized using their own attributes, inner text, and immediate[PARENT] tag—identical sibling elements without distinguishing attributes or text cannot be disambiguated visually.

Folate is released under the Apache 2.0 License.