{"slug": "show-hn-folate-local-dom-selector-model-that-beats-7b-vlms-at-1-60th-the-size", "title": "Show HN: Folate – Local DOM selector model that beats 7B VLMs at 1/60th the size", "summary": "A developer released Folate, an Apache 2.0-licensed DOM element ranker fine-tuned from google/electra-base-discriminator on the osunlp/Mind2Web dataset and quantized to INT8 ONNX at roughly 107 MB, which the author says beats 7B vision-language models at 1/60th the size for browser automation. Folate runs on CPU via onnxruntime and tokenizers with no PyTorch or transformers at inference, using about 180–250 MB of RAM, and scores (natural-language query, serialized DOM element) pairs in a single batched forward pass to return CSS/XPath selectors that are guaranteed to exist on the live page. The package is installable via pip and ships Playwright examples for grounding elements on google.com, amazon.com and github.com.", "body_md": "[PyPI](https://pypi.org/project/folate/) •\n  [Hugging Face](https://huggingface.co/figranium/folate) •\n  [Demo](https://huggingface.co/spaces/figranium/folate-demo) •\n  [Apache 2.0](https://github.com/figranium/folate/blob/main/LICENSE)\n\nA lightweight, CPU-efficient neural DOM element ranker built for browser automation within tight memory constraints (<2 GB total footprint alongside a headless browser).\n\nFine-tuned from [`google/electra-base-discriminator`](https://huggingface.co/google/electra-base-discriminator) on [`osunlp/Mind2Web`](https://huggingface.co/datasets/osunlp/Mind2Web) and dynamically quantized to **INT8 ONNX (~107 MB)**, Folate runs purely via `onnxruntime` and `tokenizers`—requiring no PyTorch or `transformers` at inference time.\n\nFolate operates as a discriminative cross-encoder that scores `(natural_language_query, serialized_dom_element)` pairs in a single batched forward pass on CPU.\n\n| Variant / Artifact | Backbone | Precision / Format | Disk Size | Runtime RAM | \n|---|---|---|---|---|\n| **`folate-int8` (Default)** | `google/electra-base-discriminator` | Dynamic `QUInt8` ONNX | ~107 MB | ~180–250 MB | \n| **`folate-base`** | `google/electra-base-discriminator` | `float32` Safetensors | ~418 MB | ~1.4+ GB (PyTorch) | \n\n- **Zero-Hallucination Selector Grounding** — Ranks real interactive DOM nodes extracted via`lxml` XPath (`<a>` ,`<button>` ,`<input>` ,`<select>` ,`<textarea>` , ARIA interactive roles, and`onclick` /`tabindex` elements), guaranteeing every returned CSS/XPath selector exists on the live page.\n- **Compact Structured Serialization** — Serializes each element into a concise`96` -token representation:`[TAG] ... [ID/NAME/TYPE/ROLE/ARIA-LABEL/PLACEHOLDER/TITLE/ALT/HREF/DATA-TESTID] ... [TEXT] ... [PARENT] ...`\n- **Edge & Low-Memory Deployment** — Designed to run alongside Chromium on resource-constrained laptops and containers without GPU acceleration.\n\n```\npip install folate\n```\n\nOr install from source:\n\n```\ngit clone https://github.com/figranium/folate\ncd folate\npython3 -m venv .venv\nsource .venv/bin/activate\npython -m pip install -r requirements.txt\n```\n\nTo run the live browser examples with Playwright:\n\n```\npython -m pip install -r examples/requirements.txt\npython -m playwright install chromium\n```\n\n`FolateSelectorFinder` automatically downloads and caches [`figranium/folate`](https://huggingface.co/figranium/folate) on first run (or loads `./folate-int8` if present locally):\n\n``` python\nfrom folate import FolateSelectorFinder\n\nfinder = FolateSelectorFinder()\nmatches = finder.rank(\"click Search Flights button\", html_source, top_k=3)\n\nfor match in matches:\n    print(f\"{match['score']:.4f} | {match['selector']} | {match['element']}\")\n```\n\nTo load weights from a custom local directory:\n\n```\nfinder = FolateSelectorFinder(model_dir=\"./folate-int8\")\n```\n\nIf you want to pre-fetch the quantized model weights via `huggingface_hub`:\n\n```\nhf download figranium/folate --local-dir ./folate-int8\nfolate/\n├── folate/\n│   ├── __init__.py\n│   └── ranker.py                  # Lightweight CPU inference engine & DOM serializer\n├── scripts/\n│   ├── prepare_data.py            # Streaming Mind2Web serializer (lxml)\n│   ├── train.py                   # Fine-tuning script (bf16/fp16/CPU)\n│   ├── export_onnx.py             # FP32 ONNX export + INT8 dynamic quantization\n│   ├── eval_mind2web.py           # Official Mind2Web evaluator (Precision/Recall/F1, Top-1/3/5 & MRR)\n│   └── eval_precision.py          # Playwright DOM-resolved selector Precision & Recall benchmark\n├── examples/\n│   ├── test_google.py             # Live Playwright search box grounding on google.com\n│   ├── test_amazon.py             # E-commerce element ranking on amazon.com (+ offline snapshot)\n│   ├── test_github.py             # Repository navigation grounding on github.com\n│   └── requirements.txt           # Optional Playwright dependency for live browser tests\n├── requirements.txt               # Lightweight CPU runtime dependencies\n├── requirements-train.txt         # Training & ONNX export dependencies\n└── LICENSE\n```\n\nFolate is evaluated on the three standard generalization splits of **Mind2Web** (`test_task`, `test_website`, and `test_domain`) across **6,037 action steps** on real-world webpages.\n\n| Split | Description | Steps | Avg Candidates | Top-1 | Top-3 | Top-5 | MRR | Avg Latency (2 vCPU) | \n|---|---|---|---|---|---|---|---|---|\n| **`test_task`** | Unseen tasks on seen websites | 1,256 | 50.0 | **93.31%** | 97.61% | 98.33% | 0.9566 | 2,309.9 ms/step | \n| **`test_website`** | Completely unseen websites | 955 | 50.0 | **88.90%** | 95.92% | 96.65% | 0.9274 | 4,328.2 ms/step | \n| **`test_domain`** | Unseen domains & verticals | 3,826 | 49.8 | **92.08%** | 97.28% | 98.61% | 0.9491 | 2,431.0 ms/step | \n\n- **Task Formulation:** At each step, the model receives the natural-language task instruction, action history context, and up to**50 candidate DOM elements** (`Avg Candidates ≈ 50.0` ) extracted from the page snapshot, and outputs a relevance logit per candidate.\n- **Metrics:**  - **Top-1 / Top-3 / Top-5:** Percentage of steps where a ground-truth target node (`pos_candidates` ) appears within the top*k* ranked elements.\n  - **Precision / Recall / F1:** Selective Top-1 precision (`TP / (TP + FP)` ) and recall (`TP / Total` ) across confidence probability thresholds (`--confidence-threshold` ,`--thresholds` ), plus candidate-level binary precision/recall.\n  - **MRR (Mean Reciprocal Rank):** Average reciprocal rank for the highest-ranked ground-truth element across all evaluated steps.\n- **Hardware & Runtime:** Latency is measured end-to-end per step on**2 vCPUs** with**no GPU acceleration** , using the exported INT8 quantized ONNX artifact (`onnxruntime` ). Variance between`test_website` (`4,328.2 ms/step` ) and`test_domain` (`2,431.0 ms/step` ) reflects longer tokenized DOM attribute sequences on unseen site templates rather than candidate count.\n\nFor context against a fast, cost-efficient frontier LLM, **GPT-6 Luna** (via OpenAI's Decisions API) scores **81.8%** on **Mind2Web Multimodal** (Google Gemini Decision 1 evaluation, Nov 2026) and **81.2%** on **Mind2Web** in the Decision Index Vision suite (Nace.AI, Nov 2026), with ~326 ms median cloud API latency and ~$0.10 per 1,000 decisions (or $0.10 / $0.50 per 1M input/output tokens in standard API usage).\n\n| Model | Type | Mind2Web Accuracy | Deployment & Runtime | Cost | \n|---|---|---|---|---|\n| **Folate (`folate-int8`)** | Specialized DOM Cross-Encoder (~107 MB) | **88.90% – 93.31%** Top-1 (91.83% overall across 6,037 steps) | 100% local CPU ( `onnxruntime` , 180–250 MB RAM) | **$0** (self-hosted) | \n| **GPT-6 Luna** | Frontier LLM (Decisions API) | 81.2% – 81.8% (Mind2Web Multimodal / Vision) | Cloud API (~326 ms median latency) | ~$0.10 / 1k decisions | \n\n**Note:** Folate is evaluated on text-based Mind2Web DOM candidates (50 candidates/step), whereas GPT-6 Luna's 81.2%–81.8% scores are on multimodal/screenshot-based Mind2Web action selection. While GPT-6 Luna is a general-purpose frontier model capable of broad reasoning and multi-step planning, Folate's task-specific ELECTRA cross-encoder achieves higher single-step DOM element grounding accuracy locally on CPU with zero API cost, zero network dependency, and zero selector hallucination.\n\nRun any of the included live browser examples from the repository root:\n\n```\npython examples/test_google.py\npython examples/test_amazon.py --offline\npython examples/test_github.py\n```\n\nTo reproduce data preparation, fine-tuning, INT8 ONNX export, and evaluation from scratch on [`osunlp/Mind2Web`](https://huggingface.co/datasets/osunlp/Mind2Web):\n\n```\npython -m pip install -r requirements-train.txt\n\n# 1. Stream and serialize Mind2Web pairs\npython scripts/prepare_data.py --output-dir ./data\n\n# 2. Fine-tune ELECTRA cross-encoder\npython scripts/train.py --data-dir ./data --save-dir ./folate-base\n\n# 3. Export to FP32 ONNX and quantize to INT8 (~107 MB)\npython scripts/export_onnx.py --model-dir ./folate-base --output-dir ./folate-int8\n\n# 4. Evaluate Mind2Web test splits (Precision, Recall, F1, Top-1/3/5 & MRR)\npython scripts/eval_mind2web.py --confidence-threshold 0.5 --json-out mind2web_results.json\n\n# 5. Run Playwright DOM-resolved selector Precision & Recall benchmark\npython scripts/eval_precision.py --confidence-threshold 0.5 --json-out precision_results.json\n```\n\n- **Intended Use:** Single-step natural-language UI element grounding (`click` ,`type` ,`select` ,`hover` ) in local browser automation pipelines where RAM and CPU budgets are constrained.\n- **Limitations:**  - **No Multi-Step Planning:** Folate ranks candidate elements for a single action query; it does not plan multi-step workflows or parse unstructured page content.\n  - **Local DOM Context Only:** Elements are serialized using their own attributes, inner text, and immediate`[PARENT]` tag—identical sibling elements without distinguishing attributes or text cannot be disambiguated visually.\n\nFolate is released under the [Apache 2.0 License](https://github.com/figranium/folate/blob/main/LICENSE).", "url": "https://wpnews.pro/news/show-hn-folate-local-dom-selector-model-that-beats-7b-vlms-at-1-60th-the-size", "canonical_source": "https://github.com/figranium/folate", "published_at": "2026-11-15 15:55:36+00:00", "updated_at": "2026-11-15 16:23:44.650867+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "machine-learning", "natural-language-processing", "developer-tools"], "entities": ["Folate", "figranium", "google/electra-base-discriminator", "osunlp/Mind2Web", "onnxruntime", "Playwright", "Hugging Face", "PyPI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-folate-local-dom-selector-model-that-beats-7b-vlms-at-1-60th-the-size", "markdown": "https://wpnews.pro/news/show-hn-folate-local-dom-selector-model-that-beats-7b-vlms-at-1-60th-the-size.md", "text": "https://wpnews.pro/news/show-hn-folate-local-dom-selector-model-that-beats-7b-vlms-at-1-60th-the-size.txt", "jsonld": "https://wpnews.pro/news/show-hn-folate-local-dom-selector-model-that-beats-7b-vlms-at-1-60th-the-size.jsonld"}}