{"slug": "did-that-job-close-or-did-your-salary-filter-hide-it", "title": "Did That Job Close, or Did Your Salary Filter Hide It?", "summary": "A developer documented a retrieval-pipeline pitfall in which a job listing's disappearance from URL lookup results is wrongly treated as proof the role closed, when the lookup's own salary and category filters may have suppressed it. The writeup proposes separating URL resolution from preference matching, preserving the last known record with a freshness warning on unresolved checks, and building resolution payloads from scratch rather than reusing search payloads. It also includes a Python URL canonicalization check that rejects malformed or tracking-laden links so they cannot trigger archival.", "body_md": "A saved job can disappear from a URL check because your own salary filter rejected it.\n\nYou're about to connect `requested - returned` to your monitoring agent's archive action. The small dread: the next notification could confidently announce that an open role has closed.\n\nLab note 028: can a URL lookup justify that transition? The catch arrives early: our job Actor applies salary and category filters during `byJobUrls` resolution. A successfully resolved listing can still be skipped.\n\n**A missing record from filtered `byJobUrls` results does not establish that a specific job closed.**\n\nThe operation combines finding a listing with deciding whether to emit it. Those are different questions, even when they share an input payload.\n\nFor an LLM pipeline, the dangerous step happens afterward: your application converts absence into a factual statement, saves that statement, and retrieves it as evidence later. Now the model has a citation for your inference. Splendid paperwork. Wrong conclusion.\n\nWe'd give the pipeline separate lanes:\n\n``` php\nsaved URL -> resolution check -> resolved / unresolved\n                                    |\n                                    v\n                             preference check\n                                    |\n                                    v\n                         matched / unresolved match\n\nindependent closure evidence -> archive permission\n```\n\nAn unresolved check should preserve the last known record with its freshness warning. It shouldn't manufacture a fresh “closed” chunk.\n\nMalformed links, nonexistent listings, and closed listings are skipped. That makes input hygiene part of lifecycle safety: a damaged saved URL must not become evidence about the employer's hiring decision.\n\nThis filter accepts a conservative subset of canonical listing URLs. It deliberately sends unfamiliar forms to review rather than attempting clever repairs.\n\nExample saved input for the checks below, without any claim that this listing is currently open:\n\n``` python\nimport re\nfrom urllib.parse import urlsplit\n\nsaved_urls = [\n    \"https://himalayas.app/companies/sola-payments/jobs/senior-data-platform-engineer\"\n]\n\ndef canonical_saved_url(value):\n    if not isinstance(value, str):\n        return None\n    try:\n        parts = urlsplit(value)\n    except ValueError:\n        return None\n\n    path_ok = re.fullmatch(\n        r\"/companies/[a-z0-9_-]+/jobs/[a-z0-9_-]+\",\n        parts.path,\n    )\n    canonical = f\"https://himalayas.app{parts.path}\"\n    if (\n        not path_ok\n        or parts.query\n        or parts.fragment\n        or value != canonical\n    ):\n        return None\n    return canonical\n\nchecked = [(value, canonical_saved_url(value)) for value in saved_urls]\naccepted_urls = list(dict.fromkeys(url for _, url in checked if url))\nrejected_urls = [value for value, url in checked if url is None]\n```\n\nPassing this check establishes structure, not availability. A rejected tracking link or unfamiliar slug belongs in `rejected_urls`, where it can't trigger archival.\n\nDeduplicating accepted inputs also prevents repeated checks from producing competing lifecycle decisions for the same saved URL.\n\nA salary floor, ceiling, or currency filter excludes listings without published pay. Category filtering can also suppress a resolved listing that doesn't match.\n\nThat creates a particularly unpleasant retrieval bug: changing a user's preferences can delete the underlying job evidence. Their preferences changed; the employer's listing didn't necessarily change with them.\n\nThe resolution payload should therefore be constructed from scratch. Copying your search payload and changing `mode` carries too much baggage.\n\nThis builder omits preferences and makes the emission allowance cover its batch:\n\n``` python\nEMIT_LIMIT = 5000\n\ndef lookup_input(batch):\n    if not 1 <= len(batch) <= EMIT_LIMIT:\n        raise ValueError(\"Batch must fit within the emitted-record limit.\")\n    run_input = {\n        \"mode\": \"byJobUrls\",\n        \"jobUrls\": batch,\n        \"maxItems\": len(batch),\n    }\n    return run_input\n```\n\nThis removes a source of ambiguity. It doesn't turn a missing result into a closure certificate.\n\nWe'd also keep preference decisions in application metadata rather than embedding them into the job description. Otherwise, retrieval can confuse “outside your salary preferences” with a property of the listing's availability.\n\nThe default `maxItems` is 20. It limits emitted job records, so a larger saved set can outgrow the allowance before you know anything about its unmatched members.\n\nA batch difference then becomes a mixture of lifecycle uncertainty and your own output budget. Asking an LLM to explain that difference only gives the uncertainty better grammar.\n\nThis collector splits the accepted URLs and uses the builder above. Preference keys are permitted only for a separate matching pass:\n\n``` python\nimport os\nfrom apify_client import ApifyClient\n\nclient = ApifyClient(os.environ[\"APIFY_TOKEN\"])\nPREFERENCE_KEYS = {\"minSalary\", \"maxSalary\", \"salaryCurrency\", \"categories\"}\n\ndef fetch_jobs(urls, preferences=None):\n    preferences = {} if preferences is None else preferences\n    if set(preferences) - PREFERENCE_KEYS:\n        raise ValueError(\"Unexpected preference key.\")\n\n    rows = []\n    for offset in range(0, len(urls), EMIT_LIMIT):\n        batch = urls[offset:offset + EMIT_LIMIT]\n        run_input = lookup_input(batch)\n        run_input.update(preferences)\n        run = client.actor(\n            \"vexaloom-labs/himalayas-remote-job-finder-with-salary-and-tech\"\n        ).call(run_input=run_input)\n        if run[\"status\"] != \"SUCCEEDED\":\n            continue\n        rows.extend(\n            client.dataset(run[\"defaultDatasetId\"]).iterate_items()\n        )\n    return rows\n```\n\nAn unsuccessful batch contributes no positive resolution evidence. An exception stops this program before it constructs archive candidates.\n\nStreaming dataset retrieval also avoids making a dataset page boundary look like another missing-job event.\n\nThe status message reports how many URLs were closed or invalid. It doesn't establish a closure reason for every missing URL.\n\nThat distinction matters when an agent writes explanations. An aggregate diagnostic can't support “your saved role closed” unless you've evidence connecting that reason to that identity.\n\n| Evidence available | Unsafe conclusion | Safe application action | \n|---|---|---|\n| Locally rejected URL | The role closed | Review the saved address | \n| Missing from preference results | The role closed | Leave lifecycle unchanged | \n| Missing from a capped batch | The role closed | Correct the batch allowance | \n| Aggregate closed-or-invalid count | This particular role closed | Keep its reason unresolved | \n\nFor reconciliation, use `jobGuid`, the canonical listing URL, rather than trying to identify jobs by title or slug.\n\nThis filter accepts only an unambiguous requested identity whose `jobUrl` agrees with it:\n\n``` python\ndef index_rows(rows, requested):\n    buckets = {}\n    for row in rows:\n        guid = canonical_saved_url(row.get(\"jobGuid\"))\n        if guid in requested:\n            buckets.setdefault(guid, []).append(row)\n\n    return {\n        guid: matches[0]\n        for guid, matches in buckets.items()\n        if len(matches) == 1\n        and canonical_saved_url(matches[0].get(\"jobUrl\")) == guid\n    }\n\nresolution_rows = fetch_jobs(accepted_urls)\nresolved = index_rows(resolution_rows, set(accepted_urls))\nunresolved_urls = set(accepted_urls) - set(resolved)\n```\n\nDuplicate identities stay unresolved here instead of silently choosing whichever record arrived last. Unexpected identities never enter the saved-job index.\n\nNeither condition deserves a closure story.\n\nReimplementing salary and category matching locally has its own traps. Annualized pay still needs currency context, and category matching includes parent categories and tech tags.\n\nFor exact Actor matching, we'd use a separate filtered pass and reconcile it locally. This costs another lookup pass, but avoids inventing an equivalent classifier from partial fields.\n\nThe combined filter below creates application-owned state keys. Positive preference results establish a match; absence remains unresolved because a later lookup can fail independently.\n\nExample preferences for that separate pass:\n\n```\npreference_filters = {\n    \"minSalary\": 120000,\n    \"salaryCurrency\": \"USD\",\n    \"categories\": [\"python\"],\n}\n\npreference_rows = fetch_jobs(list(resolved), preference_filters)\nmatched = index_rows(preference_rows, set(resolved))\n\nmonitor = {}\nfor url in accepted_urls:\n    preference_state = \"not_checked\"\n    if url in resolved:\n        preference_state = \"matched\" if url in matched else \"unresolved\"\n\n    monitor[url] = {\n        \"lifecycle_state\": \"resolved\" if url in resolved else \"unresolved\",\n        \"preference_state\": preference_state,\n    }\n```\n\nThat distinction lets your agent say, “The listing resolved, but this check didn't establish a preference match.”\n\nIt also gives retrieval a useful boundary: preference uncertainty can affect recommendations without erasing the source record. Keep check metadata separate from description chunks so it doesn't get copied across every chunk or outlive its own freshness.\n\nWe'd use this operation to resolve saved listings. A missing row can't authorize closure.\n\nThe worked example is our [Himalayas Remote Job Finder with Salary & Tech Stack](https://apify.com/vexaloom-labs/himalayas-remote-job-finder-with-salary-and-tech). Collection supplies evidence; your application owns the lifecycle transition.\n\nThis final guard requires a separate source of confirmed closure evidence and blocks archival when the current resolution returned the job:\n\n``` python\nconfirmed_closed_urls = frozenset()\n\ndef can_archive(url, closure_evidence):\n    return url in closure_evidence and url not in resolved\n\narchive_candidates = {\n    url for url in accepted_urls\n    if can_archive(url, confirmed_closed_urls)\n}\n```\n\nPopulate that evidence set through an independently verified closure process. Neither the model nor a batch-difference handler should fill it by inference.\n\nRepeated absence can justify retry or review. Repeating an ambiguous check doesn't make its explanation specific.\n\n`minSalary`, `salaryCurrency`, and `categories`.` jobGuid` values. Keep unmatched inputs unresolved in both checks.\nExample filtered input, with a placeholder listing whose current availability and preference match aren't asserted:\n\n```\n{\n  \"mode\": \"byJobUrls\",\n  \"jobUrls\": [\n    \"https://himalayas.app/companies/sola-payments/jobs/senior-data-platform-engineer\"\n  ],\n  \"minSalary\": 120000,\n  \"salaryCurrency\": \"USD\",\n  \"categories\": [\"python\"],\n  \"maxItems\": 1\n}\n```\n\n", "url": "https://wpnews.pro/news/did-that-job-close-or-did-your-salary-filter-hide-it", "canonical_source": "https://dev.to/vexaloom-labs/did-that-job-close-or-did-your-salary-filter-hide-it-462m", "published_at": "2026-10-30 07:16:53+00:00", "updated_at": "2026-10-30 07:19:39.045968+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools"], "entities": ["Himalayas"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/did-that-job-close-or-did-your-salary-filter-hide-it", "markdown": "https://wpnews.pro/news/did-that-job-close-or-did-your-salary-filter-hide-it.md", "text": "https://wpnews.pro/news/did-that-job-close-or-did-your-salary-filter-hide-it.txt", "jsonld": "https://wpnews.pro/news/did-that-job-close-or-did-your-salary-filter-hide-it.jsonld"}}