{"slug": "deterministic-tenant-scoped-resolution-of-company-jargon-to-canonical-entities", "title": "Deterministic, tenant-scoped resolution of company jargon to canonical entities", "summary": "Lexiqr, a new pip-installable Python library, resolves tenant-specific jargon to canonical entities deterministically, with typo tolerance and multilingual support. The library, requiring Python 3.10+ and using RapidFuzz, maps terms like 'flooff' to entities with character spans and score tiers, and includes CLI tools for validation and testing. It aims to replace hardcoded synonym tables and LLM guessing with an explainable, tenant-scoped resolution layer.", "body_md": "**Deterministic, tenant-scoped resolution of company jargon to canonical entities.**\n\n[Quickstart](#quickstart) • [Usage](#usage) • [Docs](/bmeunier1974/lexiqr/blob/main/docs) • [Contributing](/bmeunier1974/lexiqr/blob/main/CONTRIBUTING.md)\n\nEvery tenant calls the same thing something different. A tenant writes a lexicon\nmapping their private word — `flooff`\n\n— to one of your canonical entities:\n`product`\n\n. lexiqr loads that lexicon and turns free-form prompts into identified\nmatches, each carrying the character span it covers, its score tier, and any typo\nit corrected.\n\nTeams usually solve this by hardcoding synonym tables, retraining embeddings, or letting an LLM guess. lexiqr is a pip-installable resolution layer instead: deterministic, explainable, and scoped to one tenant.\n\nRequires **Python 3.10+**. The wheel is pure Python with a single runtime\ndependency ([RapidFuzz](https://github.com/rapidfuzz/RapidFuzz)):\n\n```\npip install lexiqr\n```\n\nor, with [uv](https://docs.astral.sh/uv/):\n\n```\nuv add lexiqr\n```\n\nA lexicon maps one tenant's private jargon to canonical entities. Here a German\ntenant maps **flooff** to the `product`\n\nentity — this is the file the examples\nbelow run against:\n\n```\n{\n  \"schemaVersion\": \"1\",\n  \"defaultLocale\": \"de-DE\",\n  \"entities\": {\n    \"product\": {\n      \"locales\": {\n        \"de-DE\": { \"preferred\": { \"singular\": \"flooff\" } }\n      }\n    }\n  }\n}\n```\n\nNow resolve a prompt. Typo tolerance is on by default, so `floof`\n\nstill resolves\nand the match names what was typed:\n\n``` python\nfrom lexiqr import EntityResolver\n\nresolver = EntityResolver.from_file(\"lexicon.json\")\n\n# \"flooff\" resolves to the product entity, with its character span and tier.\nmatch = resolver.transform(\"wo ist flooff\", locale=\"de-DE\").matches[0]\nprint(\n    f\"{match.canonical_id} <- {match.surface_form!r} at {match.span}, tier {match.score_tier.value}\"\n)\n\n# The typo \"floof\" still resolves; the match names what was typed.\ntypo = resolver.transform(\"wo ist floof\", locale=\"de-DE\").matches[0]\nprint(f\"corrected {typo.correction!r} -> {typo.surface_form!r}\")\nphp\nproduct <- 'flooff' at (7, 13), tier preferred\ncorrected 'floof' -> 'flooff'\n```\n\nLexicon authors don't need Python. The same lexicon checks and runs from the\ncommand line, so `lexiqr validate`\n\nconfirms the file is well-formed —\n\n```\nlexiqr validate lexicon.json\nlexicon.json: valid lexicon.\n```\n\n— and `lexiqr try`\n\nresolves a prompt against it, showing the same match the\ndeveloper sees:\n\n```\nlexiqr try lexicon.json --locale de-DE \"wo ist flooff\"\nprompt: \"wo ist [flooff]\"\nresolved via: de-DE\n1 match:\n\n  [1] product ← \"flooff\"\n      tier: preferred   locale: de-DE   text: \"flooff\"\n```\n\nNote\n\nThose blocks are the test suite. CI extracts them from this file, runs them, and compares the output to what you just read, so the quickstart cannot drift from the shipped API.\n\n**Deterministic by contract.** The same lexicon, prompt, and configuration produce an identical result across runs, platforms, and Pythons — a tested guarantee, not an aspiration.**Typo-tolerant, and explainable about it.** Edit budgets scale with word length, and every fuzzy match carries the correction it applied. Turn the pass off with`fuzzy=False`\n\n.**Multilingual**, with per-locale surface forms and fallback chains. Latin scripts match accent-insensitively; Arabic matches script-preserving.**Spans you can trust**— character offsets index the text the user typed, never a normalized copy.- Several terms can resolve to one entity, each carrying a\n**tenant-defined filter** verbatim. A match report replaces the per-tenant lookup table in your service. **Typed, tested, bounded**:`py.typed`\n\nand strict-mypy clean, a round-tripping report serialization, documented input limits, and a CI-enforced performance envelope.`lexiqr validate`\n\nand`lexiqr try`\n\nwork**without writing Python**, with exit codes a script can branch on.\n\nThe quickstart above is the five-minute story. This is the long one: a single command that resolves a realistic tenant lexicon and prints twelve narrated sections, each stating a claim, showing what lexiqr produced, and asserting it.\n\n```\nuv run python examples/demo.py\n```\n\nIt **exits non-zero**, naming the section that failed, so it is a verification and\nnot a brochure. It reads [ examples/medien.lexicon.json](/bmeunier1974/lexiqr/blob/main/examples/medien.lexicon.json),\na German media tenant. The run itself —\n\n[— is one flat file you can read top to bottom and copy a section out of, importing nothing but lexiqr and the standard library.](/bmeunier1974/lexiqr/blob/main/examples/demo.py)\n\n`examples/demo.py`\n\n**What the twelve sections claim**\n\n- a tenant lexicon loads from a file — validation\n*is*construction (C2) - a rejected lexicon names the entry, locale and field at fault (C3)\n- an exact match reports its entity, surface form, span into the original prompt, and score tier (C4)\n- preferred, alternate and canonical tiers each resolve and each name their tier (C4)\n- two entries resolve to one\n`product`\n\n, each reporting its own entry ID and filter (C19) - a typo resolves and carries its correction; the same prompt with\n`fuzzy=False`\n\ndoes not (C5) - a prompt in an undeclared locale variant resolves through the fallback chain, and the report names the locale that answered (C6)\n- accented and unaccented spellings both match with spans still on the typed text; Arabic matches script-preserving (C7)\n- two entities in one prompt come back ordered by position, and an overlap resolves to the longest span (C4)\n- a prompt over the documented maximum length is refused before any matching; whitespace-only is an empty report, not an error (C8)\n- a report round-trips through the canonical serialization and serializes byte-identically twice (C9)\n- the same lexicon through\n`lexiqr validate`\n\nand`lexiqr try`\n\n, with the exit codes a script reads (C14, C15)\n\nA transcript this long should not be read as the whole guarantee. Three claims it deliberately does not make, each owned by a gate of its own:\n\n**the performance envelope**— owned by the CI perf gate (`uv run pytest -m perf`\n\n). See[Performance envelope](#performance-envelope)below.**cross-platform report equality**— owned by the report-equality matrix job, which compares every OS and Python against`scripts/report_equality.golden.json`\n\n.**installability from PyPI**— owned by the release workflow's clean-virtualenv leg, which installs the published wheel and runs against it.\n\nBelow is an abridged excerpt of the real output — sections 5 and 6, with the\nother ten elided. The full transcript is committed as\n[ examples/demo.golden.txt](/bmeunier1974/lexiqr/blob/main/examples/demo.golden.txt), and the test suite\ncompares the command's output to it:\n\n```\nlexiqr — a sample run: every claim printed, every claim asserted.\nlexicon: examples/medien.lexicon.json\n\n--- 5. Two entries resolve to one entity, each with its own filter [C19] ---\n\nmatch     \"zeig mir die filme\" → product ← \"filme\"  span=(13, 18)  tier=preferred\n          locale=de-DE  entry=movie  filter={genre=drama|thriller, productType=Movie}\nmatch     \"zeig mir die serien\" → product ← \"serien\"  span=(13, 19)  tier=preferred\n          locale=de-DE  entry=series  filter={episodic=true, productType=Series}\n\n--- 6. A typo resolves and carries its correction; with fuzzy off it does not [C5] ---\n\nprompt    \"zeig mir die flme\"\ntolerant  product ← \"filme\"  span=(13, 17)  tier=preferred  locale=de-DE  entry=movie\n          filter={genre=drama|thriller, productType=Movie}  correction=\"flme\"\nexact     EntityResolver.from_file(..., fuzzy=False) → 0 matches, resolved via de-DE\n\nOK: every section held.\n```\n\nThe quickstart resolves one word. These are the pieces you reach for next, in the order they usually come up.\n\nValidation *is* construction. `Lexicon.from_file`\n\n(and `from_dict`\n\n) either\nreturns a lexicon lexiqr can trust or raises `ValidationError`\n\nnaming the entity,\nlocale, and field at fault. So you can check a tenant's file on the way in — at\ndeploy time, in an upload handler, in your own tests — without building a\nthrowaway resolver to find out:\n\n``` python\nfrom lexiqr import Lexicon, ValidationError\n\ntry:\n    lexicon = Lexicon.from_file(\"lexicon.json\")\nexcept ValidationError as invalid:\n    print(f\"rejected: {invalid}\")\nelse:\n    print(f\"valid: {sorted(lexicon.entries)} in {lexicon.default_locale}\")\nvalid: ['product'] in de-DE\n```\n\nA `Lexicon`\n\nyou already hold goes straight into a resolver —\n`EntityResolver(lexicon)`\n\n— so nothing is parsed or validated twice.\n\nA file that is not JSON at all raises `MalformedDocumentError`\n\n. It *is* a\n`ValidationError`\n\n, so the `except`\n\nabove already covers it. Catch it by name only\nto tell \"that file is not a lexicon document\" apart from \"that lexicon says the\nwrong thing\" — the distinction the CLI turns into its two exit codes.\n\n`EntityResolver(...)`\n\n, `from_file`\n\nand `from_dict`\n\nall accept a `fuzzy`\n\nkeyword,\ndefaulting to `True`\n\n. Pass `fuzzy=False`\n\nfor exact-only behaviour. The keyword is\npublic, semver-governed API.\n\n`transform()`\n\naccepts a prompt of at most **10,000 characters** (Unicode code\npoints), exported as `MAX_PROMPT_LENGTH`\n\n. A longer prompt raises\n`ValidationError`\n\nbefore any matching work happens, so a pasted document is\nrejected cheaply instead of taking a request thread with it. Reject or truncate\nupstream if your callers can paste arbitrary text.\n\nA single surface form is bounded too: at most **128 characters**, exported as\n`MAX_SURFACE_FORM_LENGTH`\n\n. That one is enforced when the lexicon loads rather\nthan when a prompt is matched — see\n[docs/lexicon-semantic-checks.md](/bmeunier1974/lexiqr/blob/main/docs/lexicon-semantic-checks.md). Code that\ngenerates labels should size them against the constant, not against a copy of the\nnumber.\n\nBoth limits are fixed parts of the contract, not per-call arguments or configuration knobs. Changing either is a semver-visible change.\n\n`serialize_report(report)`\n\nturns a `MatchReport`\n\ninto a **canonical** string:\nsorted keys, no insignificant whitespace, pure ASCII, and the match list in the\nreport's own order. Two byte-equal serializations mean two equal reports and\nnothing else. So you can snapshot a result in your test suite, diff two snapshots\nto see real behaviour change, or store one and compare it months later.\n`deserialize_report(text)`\n\nis its inverse — the form round-trips.\n\n``` python\nfrom lexiqr import serialize_report, deserialize_report\n\nsnapshot = serialize_report(resolver.transform(\"wo ist flooff\", locale=\"de-DE\"))\n# ... store `snapshot`, compare it later, or check it into your tests\n```\n\nBoth functions are public, semver-governed API. The serialized shape can only change on a major release, so a patch or minor upgrade never silently invalidates a stored snapshot.\n\nlexiqr resolves one tenant's lexicon per resolver and deliberately ships **no**\ntenant registry. Mapping tenants to resolvers is your composition, not lexiqr's,\nwhich keeps it a thin layer you control. The recipe is a cache of resolvers keyed\nby tenant, each built once:\n\n```\n# Illustrative recipe — not run in CI. Adapt the loader and cache to your stack.\nfrom functools import lru_cache\nfrom pathlib import Path\n\nfrom lexiqr import EntityResolver, MatchReport\n\n@lru_cache(maxsize=None)\ndef resolver_for(tenant_id: str) -> EntityResolver:\n    \"\"\"One resolver per tenant, built once and reused across requests.\"\"\"\n    lexicon = Path(\"lexicons\") / f\"{tenant_id}.lexicon.json\"\n    return EntityResolver.from_file(lexicon)\n\ndef resolve(tenant_id: str, prompt: str, locale: str) -> MatchReport:\n    return resolver_for(tenant_id).transform(prompt, locale)\n```\n\nA resolver is built once and then only read, so one instance per tenant is safe\nto share across requests. Size the cache to your tenant count, or swap\n`lru_cache`\n\nfor whatever eviction your deployment already uses.\n\nlexiqr is built to sit in a request path, so its performance is a stated,\nCI-enforced contract. Both numbers are measured against the seeded\n**1,000-surface-form** benchmark lexicon:\n\n`transform()`\n\np95 < 10 ms**initialization < 1 second**(cold)\n\n**How it is measured**, so you can reproduce it: initialization is timed cold —\none resolver built once, nothing warmed. For `transform()`\n\n, a fixed set of\nwarm-up calls is discarded, then p95 is taken over a fixed number of timed\niterations. A long-but-under-limit prompt is measured too, so the\n10,000-character limit is the only performance cliff rather than a hidden one\nbefore it.\n\n**The gate is not the guarantee.** The numbers above are the guarantee. The CI\nperf gate asserts that envelope times a **3× headroom factor** (p95 < 30 ms, init\n< 3 s) on a single fixed runner. Shared CI runners are noisy, and the headroom\nturns that noise into a re-run rather than a false failure. Matching has to get\nroughly an order of magnitude slower to trip the gate, so catching subtle drift\nis not its job. That is why the raw timings are also recorded, un-gated, on every\nrun.\n\nlexiqr runs on **Python 3.10, 3.11, 3.12, and 3.13** and follows\n[Semantic Versioning](https://semver.org/spec/v2.0.0.html). A version constraint\nis only as trustworthy as the surface the promise covers, so that surface is\nnamed explicitly. Semver governs:\n\n**The public API**— everything exported from the`lexiqr`\n\npackage:`EntityResolver`\n\nand its`from_file`\n\n/`from_dict`\n\n/`transform`\n\nmethods, including the`fuzzy`\n\nkeyword.**The lexicon model**—`Lexicon`\n\n, the type`EntityResolver`\n\ntakes, with its validating`from_file`\n\n/`from_dict`\n\nconstructors. Under it sits`Entry`\n\n, the named set of surface forms an entity is keyed by, carrying the entity it resolves to and the filter it holds. Then`SurfaceForms`\n\n, the shape an entry holds per locale, and`Metadata`\n\n/`MetadataValue`\n\n, that filter and the values it may hold.**The structured error types**—`ValidationError`\n\nand its coordinates (`canonical_id`\n\n,`locale`\n\n,`field`\n\n), which the CLI renders verbatim, plus`MalformedDocumentError`\n\n, the subclass raised when a file is not JSON at all.**The match report types**—`MatchReport`\n\n,`EntityMatch`\n\n, and`ScoreTier`\n\n, and the fields a caller reads off them: span, tier, correction, the entry that answered, and its metadata.**The canonical report serialization**— the byte-level shape produced by`serialize_report`\n\nand consumed by`deserialize_report`\n\n.**The two documented limits**—`MAX_PROMPT_LENGTH`\n\nand`MAX_SURFACE_FORM_LENGTH`\n\n, whose values are part of the contract.\n\nA breaking change to any of these is a major-version change. Everything else —\ninternal modules, private helpers, log wording — can change in a patch. Read the\n[CHANGELOG](/bmeunier1974/lexiqr/blob/main/CHANGELOG.md) before upgrading; every release documents what changed.\n\nThis is a single-repo project: both the meta-repo (vision and blueprint) and the product repo, with all four C4 containers shipping from here as one wheel.\n\n| Path | Container | What it is |\n|---|---|---|\n`src/lexiqr/` (excl. `cli/` ) |\ncore |\nThe deterministic resolution engine and public typed API |\n`src/lexiqr/cli/` |\ncli |\n`lexiqr validate` / `lexiqr try` for lexicon authors |\n`schema/` |\nschema |\nThe versioned JSON Schema for lexicon files, plus the shared fixture corpus |\n`.github/workflows/` , `pyproject.toml` |\ndelivery |\nCI gates and tag→PyPI trusted publishing |\n\nGuides for using lexiqr:\n\n[docs/lexicon-authoring.md](/bmeunier1974/lexiqr/blob/main/docs/lexicon-authoring.md)— writing and validating a lexicon file, the`lexiqr validate`\n\n/`lexiqr try`\n\nCLI, and its scriptable exit-code contract[docs/matching-rules.md](/bmeunier1974/lexiqr/blob/main/docs/matching-rules.md)— normalization, spans, score tiers, overlap resolution and ordering: the behavior determinism makes public[docs/lexicon-semantic-checks.md](/bmeunier1974/lexiqr/blob/main/docs/lexicon-semantic-checks.md)— the complete list of checks core enforces beyond the published schema, and why[CHANGELOG.md](/bmeunier1974/lexiqr/blob/main/CHANGELOG.md)— every release, recorded by hand\n\nThe project's cross-container truth:\n\n[VISION.md](/bmeunier1974/lexiqr/blob/main/VISION.md)— problem, actors, capabilities, non-goals, constraints[CONTEXT.md](/bmeunier1974/lexiqr/blob/main/CONTEXT.md)— the project glossary[docs/adr/](/bmeunier1974/lexiqr/blob/main/docs/adr)— architecture decision records (repo shape, contracts)\n\nTwo steps, no setup document to drift out of date — [uv](https://docs.astral.sh/uv/) does the rest:\n\n```\ngit clone https://github.com/bmeunier1974/lexiqr.git && cd lexiqr\nuv sync          # creates the venv and installs lexiqr plus its dev tools\nuv run pytest    # the same suite CI runs on every push and pull request\n```\n\n[CONTRIBUTING.md](/bmeunier1974/lexiqr/blob/main/CONTRIBUTING.md) describes the pull-request gate: lint, strict\ntype-check, and tests on every supported Python. The release process, including\nthe one-time PyPI trusted-publisher registration, lives in\n[RELEASING.md](/bmeunier1974/lexiqr/blob/main/RELEASING.md). To report a security issue, see\n[SECURITY.md](/bmeunier1974/lexiqr/blob/main/SECURITY.md).\n\nMIT — see [LICENSE](/bmeunier1974/lexiqr/blob/main/LICENSE).", "url": "https://wpnews.pro/news/deterministic-tenant-scoped-resolution-of-company-jargon-to-canonical-entities", "canonical_source": "https://github.com/bmeunier1974/lexiqr", "published_at": "2026-08-20 15:16:01+00:00", "updated_at": "2026-08-20 15:45:49.391767+00:00", "lang": "en", "topics": ["developer-tools", "natural-language-processing"], "entities": ["lexiqr", "RapidFuzz", "Python"], "alternates": {"html": "https://wpnews.pro/news/deterministic-tenant-scoped-resolution-of-company-jargon-to-canonical-entities", "markdown": "https://wpnews.pro/news/deterministic-tenant-scoped-resolution-of-company-jargon-to-canonical-entities.md", "text": "https://wpnews.pro/news/deterministic-tenant-scoped-resolution-of-company-jargon-to-canonical-entities.txt", "jsonld": "https://wpnews.pro/news/deterministic-tenant-scoped-resolution-of-company-jargon-to-canonical-entities.jsonld"}}