Hi all,
I’ve been building a pipeline that needs to match “the same real-world thing” as it’s described differently across multiple independent data sources — different wording, different granularity, sometimes different structure entirely. Wanted to share the approach and see what other people have found works (or doesn’t).
Current setup:
Scan stage — pulls candidate records from several sources on a schedule
Match stage — embed everything and do similarity search to generate candidate pairs (running on 2x T4s, splitting the embedding load across both)
Verify stage — because embedding similarity alone throws too many false positives, I run a second pass through an LLM (currently) using Ollama Cloud) that looks at each candidate pair and makes a binary yes/no call with reasoning
Results feed a live dashboard
What I’m still tuning / would love input on:
Embedding model choice (x) - curious if anyone’s compared options specifically for short, noisy, informally-worded text (not clean prose)
Verification pass reliability — the LLM-as-verifier step is the biggest lever for precision right now. Anyone gone deeper on structured verification prompts vs. fine-tuning a smaller classifier instead of paying LLM cost per pair?
Throughput on dual T4s — is splitting embedding batches across two T4s actually the right move vs. a single bigger card? Curious what others’ cost/throughput tradeoffs looked like at similar scale
False positive tail — the long tail of “technically similar but semantically different” pairs is where most of my remaining errors live. Anyone found a good pattern for catching that without just throwing more LLM calls at it?
Not trying to be cagey about the domain, just keeping this focused on the ML/infra side. Happy to go into more detail on any part if useful.
Seeking wisdom