{"slug": "how-i-detect-b2b-buying-intent-in-public-developer-forums-with-python-an-llm", "title": "How I detect B2B buying intent in public developer forums with Python + an LLM", "summary": "A developer described a Python pipeline for detecting B2B buying intent in public developer and founder forums, fetching recent posts from Hacker News, Dev.to and Stack Exchange Software Recommendations, deduplicating them with SHA-256 hashes, pre-filtering with cheap keyword rules, then scoring each post 1–10 with an LLM that returns strict JSON and keeping only scores of 7 or higher. The production collector runs every six hours with a 36-hour lookback across Reddit, Hacker News, Lobsters, Dev.to and Stack Exchange, using the Arctic Shift archive as the primary Reddit path and official Reddit as fallback to avoid 403 and 429 errors.", "body_md": "Keyword alerts are a bad way to find software buyers.\n\nSearch Reddit or Hacker News for \"alternative to\" and you will get launch posts, affiliate roundups, venting, and the occasional genuine \"I need to replace this before renewal.\" The last one is the only row I want. Everything else is noise that burns time and, if you score it with a model, tokens.\n\nThis article walks through the pipeline I use to detect **B2B buying intent** in public developer and founder forums: fetch recent posts, deduplicate them, pre-filter with cheap text rules, then ask an LLM for a strict JSON score. I run the production collector in TypeScript; the approach is easier to read in Python, which is what I will use here.\n\nAt the end I will point to a hosted API that already does this, if you would rather not operate a collector yourself. The useful part is the method. You can stop after the script and still have something that works.\n\nI am not trying to detect curiosity or brand mentions. I want posts where the **author** is trying to buy, switch, or replace software: a SaaS tool, an AI product, automation, a developer tool, or a marketing/growth tool.\n\nA practical scoring scale, used as an integer from 1 to 10:\n\nTwo rules matter more than the rest of the prompt:\n\nI also ask the model for a category, an urgency (`low`, `medium`, or `high`), the software names explicitly cited, and one sentence that describes the buyer's need. The sentence is what you will actually use later, in Slack or in a CRM note.\n\n```\npublic posts\n    → SHA-256 hash of platform + external id (dedup)\n    → keyword pre-filter, plus a looser software + question pass\n    → LLM JSON analysis (capped per run and per day)\n    → keep only if intent_score >= 7\n```\n\nIn production this runs every six hours, looking back 36 hours, against Reddit, Hacker News, Lobsters, Dev.to, and Stack Exchange Software Recommendations. The Python script below uses the three sources that are easiest to call from a laptop: the HN Algolia API, the Dev.to API (`help` and `discuss`), and the Stack Exchange API for Software Recommendations. The same pre-filter and scorer apply if you add Reddit later.\n\nA note on Reddit, because it is the noisiest source and the one people reach for first: official JSON and RSS endpoints often return 403 or 429 from cloud IPs. A production collector needs a polite User-Agent, delays between requests, and a fallback. I use a near-real-time public archive (Arctic Shift) as the primary Reddit path and official Reddit as a fallback, and I skip any subreddit that errors so one 403 does not kill the run.\n\nStick to public APIs, send a real User-Agent, and sleep between pages. Software Recommendations is a special case: almost every question is \"please recommend software,\" so I treat those posts as buying-intent **candidates** even when they do not match my keyword list. They still go through the LLM. The pre-filter is skipped; the score threshold is not.\n\nHacker News is searched with quoted intent phrases (`\"alternative to\"`, `\"looking to replace\"`, `\"willing to pay\"`, and so on) across Ask HN, stories, and comments. Comments are noisy, so I drop any comment that does not also look like a software/tool discussion. Dev.to is simpler: page recent articles tagged `help` or `discuss` and keep those inside the time window.\n\n``` python\nimport time\nimport requests\n\nUSER_AGENT = \"BuyingIntentTutorial/1.0 (dev.to article)\"\nHEADERS = {\"User-Agent\": USER_AGENT, \"Accept\": \"application/json\"}\nHN_ALGOLIA = \"https://hn.algolia.com/api/v1/search_by_date\"\n\nINTENT_SEARCH_PHRASES = (\n    '\"looking for\"',\n    '\"alternative to\"',\n    '\"recommend a\"',\n    '\"any suggestions\"',\n    '\"switching from\"',\n    '\"willing to pay\"',\n    '\"need a tool\"',\n    '\"replacement for\"',\n    '\"looking to replace\"',\n    '\"frustrated with\"',\n)\n\ndef get_json(url, params=None, timeout=15):\n    res = requests.get(url, headers=HEADERS, params=params, timeout=timeout)\n    res.raise_for_status()\n    return res.json()\n\ndef fetch_hn(since_unix, until_unix, max_pages=2):\n    posts, seen = [], set()\n    for phrase in INTENT_SEARCH_PHRASES:\n        for tags in (\"ask_hn\", \"story\", \"comment\"):\n            for page in range(max_pages):\n                data = get_json(\n                    HN_ALGOLIA,\n                    {\n                        \"query\": phrase,\n                        \"tags\": tags,\n                        \"numericFilters\": f\"created_at_i>={since_unix},created_at_i<{until_unix}\",\n                        \"hitsPerPage\": 50,\n                        \"page\": page,\n                    },\n                )\n                hits = data.get(\"hits\") or []\n                for hit in hits:\n                    object_id = hit.get(\"objectID\")\n                    if not object_id or object_id in seen:\n                        continue\n                    title = (hit.get(\"title\") or hit.get(\"story_title\") or \"HN comment\").strip()\n                    body = hit.get(\"comment_text\") or hit.get(\"story_text\") or \"\"\n                    if tags == \"comment\" and not SOFTWARE_QUALIFIER.search(f\"{title}\\n{body}\"):\n                        continue\n                    seen.add(object_id)\n                    posts.append(\n                        {\n                            \"platform\": \"hackernews\",\n                            \"external_id\": str(object_id),\n                            \"source\": \"hackernews\",\n                            \"url\": f\"https://news.ycombinator.com/item?id={object_id}\",\n                            \"title\": title,\n                            \"body\": strip_html(body),\n                            \"author\": hit.get(\"author\"),\n                        }\n                    )\n                if not hits or page + 1 >= data.get(\"nbPages\", 1):\n                    break\n                time.sleep(0.15)\n    return posts\n```\n\n`SOFTWARE_QUALIFIER` and `strip_html` are in the complete script at the end. Dev.to and Software Recommendations follow the same shape: normalize each item into `platform`, `external_id`, `url`, `title`, `body`, and `author`.\n\nHash `platform` and the source's own id, not the title. Titles get edited; ids do not.\n\n``` python\nimport hashlib\n\ndef source_hash(post):\n    key = f\"{post['platform']}:{post['external_id']}\"\n    return hashlib.sha256(key.encode()).hexdigest()\n```\n\nIn production I store two things: accepted signals, and a memory of every post already analysed (including rejects). Without the second table, a post that scored 4 yesterday will be sent to the model again tomorrow. That is wasted spend, and it also hides your true yield (accepted / analysed), which you need if you ever tune keywords.\n\nFor a local script, a JSON file of hashes is enough.\n\nIf you send every Ask HN and every `#discuss` post to an LLM, most of your bill will be spent confirming that a post is not about buying software. A case-insensitive substring check against a short phrase list removes the bulk of that.\n\nPhrases that have been useful in practice:\n\n`alternative to`, `looking to replace`, `replacement for`, `switching from`, `migrate from`\n`looking for`, `recommend a`, `any suggestions`, `any recommendations`, `can anyone recommend`, `seeking recommendations`, `tool recommendations`\n`need a tool`, `need software`, `need a saas`, `software for`, `best tool`, `what tool do you`, `how do you automate`\n`willing to pay`, `ready to pay`, `budget for`, `in the market for`, `frustrated with`\nTwo exceptions keep this from being too tight:\n\n`saas`, `tool`, `crm`, `automation`, `vendor`, and similar) `?`, `alternative`, `recommend`, `replace`, Then rank: more keyword hits first, then longer bodies. Short titles with no context are a poor use of the cap.\n\n``` python\ndef match_keywords(text, keywords):\n    haystack = \" \".join(text.lower().split())\n    return [k for k in keywords if k.lower() in haystack]\n\ndef is_loose_buying_candidate(title, body):\n    text = f\"{title}\\n{body}\"\n    if not SOFTWARE_QUALIFIER.search(text):\n        return False\n    return any(cue.search(text) for cue in BUYING_QUESTION_CUES)\n\ndef to_candidates(posts, keywords):\n    keyword_hits, loose_hits = [], []\n    for post in posts:\n        if post[\"platform\"] == \"stackexchange\" and post[\"source\"] == \"softwarerecs\":\n            keyword_hits.append({**post, \"matched_keywords\": [\"softwarerecs\"]})\n            continue\n        matched = match_keywords(f\"{post['title']}\\n{post['body']}\", keywords)\n        if matched:\n            keyword_hits.append({**post, \"matched_keywords\": matched})\n        elif is_loose_buying_candidate(post[\"title\"], post[\"body\"]):\n            loose_hits.append({**post, \"matched_keywords\": [\"loose_intent\"]})\n\n    def rank(post):\n        return (-len(post[\"matched_keywords\"]), -len(post[\"body\"]))\n\n    keyword_hits.sort(key=rank)\n    loose_hits.sort(key=rank)\n    return keyword_hits + loose_hits\n```\n\nOnly then do you slice the queue to a budget (`MAX_AI_CALLS` per run, plus a daily cap if you put this on a cron).\n\nUse an OpenAI-compatible chat API at a low temperature (0.1). The exact provider does not matter. The contract does: one JSON object, no markdown, no commentary, with a fixed set of keys.\n\nCategories I use:\n\n`ai_tools`, `saas_alternative`, `automation`, `dev_tools`, `marketing_growth`\n\nThe user message is boring on purpose: platform, title, and a body truncated to 2,000 characters. Long threads do not need to be fully scored; the question is usually in the first few paragraphs.\n\n```\nCATEGORIES = {\n    \"ai_tools\",\n    \"saas_alternative\",\n    \"automation\",\n    \"dev_tools\",\n    \"marketing_growth\",\n}\nURGENCIES = {\"low\", \"medium\", \"high\"}\n\nSYSTEM_PROMPT = \"\"\"\nYou are a B2B sales-intelligence analyst. You read a public post from Reddit,\nHacker News, Lobsters, Dev.to or Stack Exchange (Software Recommendations)\nand decide whether its author shows REAL buying intent for software (SaaS, AI\ntools, automation, developer tools, marketing/growth tools): actively searching\nfor a tool, asking for alternatives, frustrated with a current vendor, ready\nto switch or pay.\n\nScoring guide for intent_score (integer 1-10):\n1-3 = no purchase intent (news, opinion, self-promotion, launch announcement,\n      hiring, rant without need)\n4-6 = vague interest or early research\n7-8 = clear need, actively evaluating solutions\n9-10 = urgent, explicit intent to buy/switch now (budget, deadline,\n       \"need it this week\")\nPosts where the author promotes their OWN product must score <= 3.\n\nRespond with ONLY one JSON object, no markdown, no commentary, exactly these keys:\n{\n  \"intent_score\": <integer 1-10>,\n  \"category\": \"ai_tools\" | \"saas_alternative\" | \"automation\" | \"dev_tools\" | \"marketing_growth\",\n  \"problem_summary\": \"<one actionable sentence in English describing the buyer's need>\",\n  \"mentioned_tools\": [\"<software/product names explicitly cited>\"],\n  \"urgency\": \"low\" | \"medium\" | \"high\"\n}\n\"\"\".strip()\n\ndef build_user_prompt(post, max_chars=2000):\n    body = post[\"body\"]\n    if len(body) > max_chars:\n        body = body[:max_chars] + \"…\"\n    source = f\" ({post['source']})\" if post.get(\"source\") else \"\"\n    return f\"Platform: {post['platform']}{source}\\nTitle: {post['title']}\\nBody: {body or '(empty)'}\"\n```\n\nModels will still wrap the object in `` ` ``` json` fences, emit smart quotes, leave a trailing comma, or add a `<think>` block. Parse defensively, then **validate**:\n\n`intent_score` must be numeric; round it and clamp it to 1–10`category` must be one of the five values above, or the whole analysis is invalid`problem_summary` must be a string of at least a few characters (I cap it at 500)`urgency` must be `low`, `medium`, or `high`; if it is missing or malformed I default it to `mentioned_tools` is a de-duplicated list of short strings (I cap each name at 60 characters and the list at 15)\nIf validation fails, retry a few times with backoff. If it still fails, skip the post and move on. Do not insert a half-parsed row.\n\n``python`\n\ndef validate_analysis(raw):\n\n    if not isinstance(raw, dict):\n\n        return None\n\n    try:\n\n        score = int(round(float(raw.get(\"intent_score\"))))\n\n    except (TypeError, ValueError):\n\n        return None\n\n    score = min(10, max(1, score))\n\n```\ncategory = str(raw.get(\"category\") or \"\").strip().lower().replace(\"-\", \"_\").replace(\" \", \"_\")\nif category not in CATEGORIES:\n    return None\n\nsummary = \" \".join(str(raw.get(\"problem_summary\") or \"\").split()).strip()\nif len(summary) < 5:\n    return None\n\nurgency = str(raw.get(\"urgency\") or \"\").strip().lower()\nif urgency not in URGENCIES:\n    urgency = \"low\"\n\ntools, seen = [], set()\nfor item in raw.get(\"mentioned_tools\") or []:\n    if not isinstance(item, str):\n        continue\n    name = item.strip()[:60]\n    key = name.lower()\n    if not name or key in seen:\n        continue\n    seen.add(key)\n    tools.append(name)\n    if len(tools) >= 15:\n        break\n\nreturn {\n    \"intent_score\": score,\n    \"category\": category,\n    \"problem_summary\": summary[:500],\n    \"mentioned_tools\": tools,\n    \"urgency\": urgency,\n}\n```\n\n`` ` ``\n\nOnly persist a signal when `intent_score >= 7`. Still record the hash of rejects, so you never pay to score them again.\n\nThis is a public Reddit thread, with the kind of structured result the scorer should return. The username is omitted on purpose.\n\n`saas_alternative`\n`[\"Veeqo\", \"Vesyl\"]`\nThat is a keep: a named current tool, a specific constraint, and an active search for a replacement. A \"what's the best AI tool?\" post with no constraint would land in the 4–6 band and be dropped.\n\nSave the complete file as `detect_intent.py`, then:\n\n``bash`\n\npip install requests openai\n\nexport OPENAI_API_KEY=sk-...\n\nexport OPENAI_MODEL=gpt-4o-mini   # or whatever your provider expects\n\npython detect_intent.py\n\n`` ` ``\n\nThe script fetches about 36 hours of HN / Dev.to / Software Recommendations posts, pre-filters, scores at most 15 candidates, and prints anything that meets the threshold. It writes analysed hashes to `processed_hashes.json`.\n\n``python`\n\n\"\"\"Detect B2B buying intent in public HN, Dev.to and Software Recs posts.\"\"\"\n\nfrom **future** import annotations\n\nimport hashlib\n\nimport json\n\nimport os\n\nimport re\n\nimport time\n\nfrom datetime import datetime, timezone\n\nfrom pathlib import Path\n\nimport requests\n\nfrom openai import OpenAI\n\nLOOKBACK_HOURS = 36\n\nMAX_AI_CALLS = 15\n\nMIN_INTENT_THRESHOLD = 7\n\nBODY_MAX_CHARS = 2000\n\nUSER_AGENT = \"BuyingIntentTutorial/1.0 (dev.to article)\"\n\nHEADERS = {\"User-Agent\": USER_AGENT, \"Accept\": \"application/json\"}\n\nHASH_FILE = Path(\"processed_hashes.json\")\n\nKEYWORDS = [\n\n    \"alternative to\",\n\n    \"looking for\",\n\n    \"recommend a\",\n\n    \"any suggestions\",\n\n    \"any recommendations\",\n\n    \"suggestions for\",\n\n    \"switching from\",\n\n    \"migrate from\",\n\n    \"budget for\",\n\n    \"willing to pay\",\n\n    \"ready to pay\",\n\n    \"need a tool\",\n\n    \"need software\",\n\n    \"need a saas\",\n\n    \"looking to replace\",\n\n    \"replacement for\",\n\n    \"frustrated with\",\n\n    \"can anyone recommend\",\n\n    \"seeking recommendations\",\n\n    \"in the market for\",\n\n    \"software for\",\n\n    \"best tool\",\n\n    \"how do you automate\",\n\n    \"what tool do you\",\n\n    \"tool recommendations\",\n\n]\n\nINTENT_SEARCH_PHRASES = (\n\n    '\"looking for\"',\n\n    '\"alternative to\"',\n\n    '\"recommend a\"',\n\n    '\"any suggestions\"',\n\n    '\"any recommendations\"',\n\n    '\"switching from\"',\n\n    '\"willing to pay\"',\n\n    '\"ready to pay\"',\n\n    '\"need a tool\"',\n\n    '\"need software\"',\n\n    '\"replacement for\"',\n\n    '\"looking to replace\"',\n\n    '\"can anyone recommend\"',\n\n    '\"in the market for\"',\n\n    '\"frustrated with\"',\n\n)\n\nCATEGORIES = {\n\n    \"ai_tools\",\n\n    \"saas_alternative\",\n\n    \"automation\",\n\n    \"dev_tools\",\n\n    \"marketing_growth\",\n\n}\n\nURGENCIES = {\"low\", \"medium\", \"high\"}\n\nSOFTWARE_QUALIFIER = re.compile(\n\n    r\"\\b(saas|software|tool|tools|app|apps|platform|crm|api|\"\n\n    r\"automat(?:e|ion|ing)?|subscription|vendor|workspace|plugin|\"\n\n    r\"integration|dashboard|productiv(?:e|ity)|analytics|billing|\"\n\n    r\"invoice|payroll|onboard(?:ing)?)\\b\",\n\n    re.I,\n\n)\n\nBUYING_QUESTION_CUES = [\n\n    re.compile(r\"\\?\"),\n\n    re.compile(r\"\\bwhat (?:do you|would you|should i|can i)\\b\", re.I),\n\n    re.compile(r\"\\bhow do (?:you|i|we)\\b\", re.I),\n\n    re.compile(r\"\\b(?:anyone|any one) (?:here )?(?:use|using|know|tried|recommend)\", re.I),\n\n    re.compile(r\"\\binstead of\\b\", re.I),\n\n    re.compile(r\"\\brecommend(?:ation)?s?\\b\", re.I),\n\n    re.compile(r\"\\balternative\\b\", re.I),\n\n    re.compile(r\"\\bwhat(?:'s| is) the best\\b\", re.I),\n\n    re.compile(r\"\\blooking to\\b\", re.I),\n\n    re.compile(r\"\\bneed (?:help finding|something|an? (?:tool|app|software|saas|platform))\", re.I),\n\n    re.compile(r\"\\bswitch(?:ing)? (?:from|to)\\b\", re.I),\n\n    re.compile(r\"\\breplace\\b\", re.I),\n\n]\n\nSYSTEM_PROMPT = \"\"\"\n\nYou are a B2B sales-intelligence analyst. You read a public post from Reddit,\n\nHacker News, Lobsters, Dev.to or Stack Exchange (Software Recommendations)\n\nand decide whether its author shows REAL buying intent for software (SaaS, AI\n\ntools, automation, developer tools, marketing/growth tools): actively searching\n\nfor a tool, asking for alternatives, frustrated with a current vendor, ready\n\nto switch or pay.\n\nScoring guide for intent_score (integer 1-10):\n\n1-3 = no purchase intent (news, opinion, self-promotion, launch announcement,\n\n      hiring, rant without need)\n\n4-6 = vague interest or early research\n\n7-8 = clear need, actively evaluating solutions\n\n9-10 = urgent, explicit intent to buy/switch now (budget, deadline,\n\n       \"need it this week\")\n\nPosts where the author promotes their OWN product must score <= 3.\n\nRespond with ONLY one JSON object, no markdown, no commentary, exactly these keys:\n\n{\n\n  \"intent_score\": ,\n\n  \"category\": \"ai_tools\" | \"saas_alternative\" | \"automation\" | \"dev_tools\" | \"marketing_growth\",\n\n  \"problem_summary\": \"\",\n\n  \"mentioned_tools\": [\"\"],\n\n  \"urgency\": \"low\" | \"medium\" | \"high\"\n\n}\n\n\"\"\".strip()\n\ndef strip_html(text: str) -> str:\n\n    text = re.sub(r\"<[^>]+>\", \" \", text or \"\")\n\n    return re.sub(r\"\\s+\", \" \", text).strip()\n\ndef get_json(url: str, params=None):\n\n    res = requests.get(url, headers=HEADERS, params=params, timeout=15)\n\n    res.raise_for_status()\n\n    return res.json()\n\ndef source_hash(post: dict) -> str:\n\n    return hashlib.sha256(f\"{post['platform']}:{post['external_id']}\".encode()).hexdigest()\n\ndef load_hashes() -> set[str]:\n\n    if HASH_FILE.exists():\n\n        return set(json.loads(HASH_FILE.read_text()))\n\n    return set()\n\ndef save_hashes(hashes: set[str]) -> None:\n\n    HASH_FILE.write_text(json.dumps(sorted(hashes), indent=2))\n\ndef fetch_hn(since: int, until: int) -> list[dict]:\n\n    posts, seen = [], set()\n\n    for phrase in INTENT_SEARCH_PHRASES:\n\n        for tags in (\"ask_hn\", \"story\", \"comment\"):\n\n            for page in range(2):\n\n                data = get_json(\n\n                    \"[https://hn.algolia.com/api/v1/search_by_date](https://hn.algolia.com/api/v1/search_by_date)\",\n\n                    {\n\n                        \"query\": phrase,\n\n                        \"tags\": tags,\n\n                        \"numericFilters\": f\"created_at_i>={since},created_at_i<{until}\",\n\n                        \"hitsPerPage\": 50,\n\n                        \"page\": page,\n\n                    },\n\n                )\n\n                hits = data.get(\"hits\") or []\n\n                for hit in hits:\n\n                    object_id = hit.get(\"objectID\")\n\n                    if not object_id or object_id in seen:\n\n                        continue\n\n                    title = (hit.get(\"title\") or hit.get(\"story_title\") or \"HN comment\").strip()\n\n                    body = hit.get(\"comment_text\") or hit.get(\"story_text\") or \"\"\n\n                    if tags == \"comment\" and not SOFTWARE_QUALIFIER.search(f\"{title}\\n{body}\"):\n\n                        continue\n\n                    seen.add(object_id)\n\n                    posts.append(\n\n                        {\n\n                            \"platform\": \"hackernews\",\n\n                            \"external_id\": str(object_id),\n\n                            \"source\": \"hackernews\",\n\n                            \"url\": f\"[https://news.ycombinator.com/item?id={object_id}](https://news.ycombinator.com/item?id=%7Bobject_id%7D)\",\n\n                            \"title\": title,\n\n                            \"body\": strip_html(body),\n\n                            \"author\": hit.get(\"author\"),\n\n                        }\n\n                    )\n\n                if not hits or page + 1 >= data.get(\"nbPages\", 1):\n\n                    break\n\n                time.sleep(0.15)\n\n    return posts\n\ndef fetch_devto(since_ms: int, until_ms: int) -> list[dict]:\n\n    posts, seen = [], set()\n\n    for tag in (\"help\", \"discuss\"):\n\n        reached_old = False\n\n        for page in range(1, 6):\n\n            items = get_json(\n\n                \"[https://dev.to/api/articles](https://dev.to/api/articles)\",\n\n                {\"per_page\": 30, \"page\": page, \"tag\": tag},\n\n            )\n\n            if not items:\n\n                break\n\n            for item in items:\n\n                item_id = item.get(\"id\")\n\n                published = item.get(\"published_at\")\n\n                posted_ms = (\n\n                    datetime.fromisoformat(published.replace(\"Z\", \"+00:00\")).timestamp() * 1000\n\n                    if published\n\n                    else None\n\n                )\n\n                if posted_ms is not None and posted_ms < since_ms:\n\n                    reached_old = True\n\n                    continue\n\n                if not item_id or item_id in seen:\n\n                    continue\n\n                if posted_ms is not None and not (since_ms <= posted_ms < until_ms):\n\n                    continue\n\n                seen.add(item_id)\n\n                username = (item.get(\"user\") or {}).get(\"username\") or \"dev\"\n\n                posts.append(\n\n                    {\n\n                        \"platform\": \"devto\",\n\n                        \"external_id\": str(item_id),\n\n                        \"source\": \"devto\",\n\n                        \"url\": item.get(\"url\") or f\"[https://dev.to/{username}/{item.get('slug](<https://dev.to/%7Busername%7D/%7Bitem.get('slug>)', item_id)}\",\n\n                        \"title\": (item.get(\"title\") or \"\").strip(),\n\n                        \"body\": (item.get(\"description\") or \"\").strip(),\n\n                        \"author\": username,\n\n                    }\n\n                )\n\n            if reached_old:\n\n                break\n\n            time.sleep(0.8)\n\n    return posts\n\ndef fetch_software_recs(since: int, until: int) -> list[dict]:\n\n    posts = []\n\n    for page in range(1, 4):\n\n        data = get_json(\n\n            \"[https://api.stackexchange.com/2.3/questions](https://api.stackexchange.com/2.3/questions)\",\n\n            {\n\n                \"order\": \"desc\",\n\n                \"sort\": \"creation\",\n\n                \"site\": \"softwarerecs\",\n\n                \"pagesize\": 50,\n\n                \"page\": page,\n\n                \"filter\": \"withbody\",\n\n                \"fromdate\": since,\n\n                \"todate\": until,\n\n            },\n\n        )\n\n        for item in data.get(\"items\") or []:\n\n            question_id = item.get(\"question_id\")\n\n            if not question_id or not item.get(\"title\"):\n\n                continue\n\n            owner = (item.get(\"owner\") or {}).get(\"display_name\")\n\n            posts.append(\n\n                {\n\n                    \"platform\": \"stackexchange\",\n\n                    \"external_id\": str(question_id),\n\n                    \"source\": \"softwarerecs\",\n\n                    \"url\": item.get(\"link\")\n\n                    or f\"[https://softwarerecs.stackexchange.com/questions/{question_id}](https://softwarerecs.stackexchange.com/questions/%7Bquestion_id%7D)\",\n\n                    \"title\": item[\"title\"].strip(),\n\n                    \"body\": strip_html(item.get(\"body\") or \"\"),\n\n                    \"author\": owner,\n\n                }\n\n            )\n\n        if not data.get(\"has_more\"):\n\n            break\n\n        time.sleep(0.8)\n\n    return posts\n\ndef match_keywords(text: str, keywords: list[str]) -> list[str]:\n\n    haystack = \" \".join(text.lower().split())\n\n    return [k for k in keywords if k.lower() in haystack]\n\ndef is_loose_buying_candidate(title: str, body: str) -> bool:\n\n    text = f\"{title}\\n{body}\"\n\n    return bool(SOFTWARE_QUALIFIER.search(text) and any(cue.search(text) for cue in BUYING_QUESTION_CUES))\n\ndef to_candidates(posts: list[dict]) -> list[dict]:\n\n    keyword_hits, loose_hits = [], []\n\n    for post in posts:\n\n        if post[\"platform\"] == \"stackexchange\" and post[\"source\"] == \"softwarerecs\":\n\n            keyword_hits.append({**post, \"matched_keywords\": [\"softwarerecs\"]}) continue\n        matched = match_keywords(f\"{post['title']}\\n{post['body']}\", KEYWORDS)\n        if matched:\n            keyword_hits.append({**post, \"matched_keywords\": matched})\n\ndef parse_json_safe(raw: str | None):\n\n    if not raw:\n\n        return None\n\n    text = re.sub(r\"[\\s\\S]*?**\", \"\", raw, flags=re.I).strip()\n    candidates = re.findall(r\"`(?:json)?\\s*([\\s\\S]*?)`\", text, flags=re.I)\n    candidates.append(text)\n    start = text.find(\"{\")\n    end = text.rfind(\"}\")\n    if start != -1 and end > start:\n        candidates.append(text[start : end + 1])\n    for candidate in candidates:\n        repaired = candidate.replace(\"“\", '\"').replace(\"”\", '\"').replace(\"’\", \"'\")\n        repaired = re.sub(r\",\\s*([}]])\", r\"\\1\", repaired)\n\n        try:\n\n            return json.loads(repaired)\n\n        except json.JSONDecodeError:\n\n            continue\n\n    return None\n\ndef validate_analysis(raw) -> dict | None:\n\n    if not isinstance(raw, dict):\n\n        return None\n\n    try:\n\n        score = min(10, max(1, int(round(float(raw.get(\"intent_score\"))))))\n\n    except (TypeError, ValueError):\n\n        return None\n\n    category = str(raw.get(\"category\") or \"\").strip().lower().replace(\"-\", \"*\").replace(\" \", \"*\")\n\n    if category not in CATEGORIES:\n\n        return None\n\n    summary = \" \".join(str(raw.get(\"problem_summary\") or \"\").split()).strip()\n\n    if len(summary) < 5:\n\n        return None\n\n    urgency = str(raw.get(\"urgency\") or \"\").strip().lower()\n\n    if urgency not in URGENCIES:\n\n        urgency = \"low\"\n\n    tools, seen = [], set()\n\n    for item in raw.get(\"mentioned_tools\") or []:\n\n        if not isinstance(item, str):\n\n            continue\n\n        name = item.strip()[:60]\n\n        if not name or name.lower() in seen:\n\n            continue\n\n        seen.add(name.lower())\n\n        tools.append(name)\n\n        if len(tools) >= 15:\n\n            break\n\n    return {\n\n        \"intent_score\": score,\n\n        \"category\": category,\n\n        \"problem_summary\": summary[:500],\n\n        \"mentioned_tools\": tools,\n\n        \"urgency\": urgency,\n\n    }\n\ndef analyse(client: OpenAI, model: str, post: dict) -> dict | None:\n\n    body = post[\"body\"]\n\n    if len(body) > BODY_MAX_CHARS:\n\n        body = body[:BODY_MAX_CHARS] + \"…\"\n\n    source = f\" ({post['source']})\" if post.get(\"source\") else \"\"\n\n    user = f\"Platform: {post['platform']}{source}\\nTitle: {post['title']}\\nBody: {body or '(empty)'}\"\n\n    completion = client.chat.completions.create(\n\n        model=model,\n\n        temperature=0.1,\n\n        max_tokens=2000,\n\n        messages=[\n\n            {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n\n            {\"role\": \"user\", \"content\": user},\n\n        ],\n\n    )\n\n    parsed = parse_json_safe(completion.choices[0].message.content)\n\n    return validate_analysis(parsed)\n\ndef main() -> None:\n\n    now = datetime.now(timezone.utc)\n\n    until = int(now.timestamp())\n\n    since = until - LOOKBACK_HOURS * 3600\n\n    since_ms, until_ms = since * 1000, until * 1000\n\n```\nposts = fetch_hn(since, until) + fetch_devto(since_ms, until_ms) + fetch_software_recs(since, until)\nknown = load_hashes()\nfresh = [p for p in posts if source_hash(p) not in known]\nqueued = to_candidates(fresh)[:MAX_AI_CALLS]\nprint(f\"fetched={len(posts)} new={len(fresh)} to_analyse={len(queued)}\")\n\nclient = OpenAI(base_url=os.getenv(\"OPENAI_BASE_URL\") or None)\nmodel = os.getenv(\"OPENAI_MODEL\") or \"gpt-4o-mini\"\nkept = 0\nfor post in queued:\n    digest = source_hash(post)\n    try:\n        analysis = analyse(client, model, post)\n    except Exception as exc:\n        print(f\"failed {post['url']}: {exc}\")\n        continue\n    known.add(digest)\n    if not analysis:\n        print(f\"invalid JSON {post['url']}\")\n        continue\n    if analysis[\"intent_score\"] < MIN_INTENT_THRESHOLD:\n        print(f\"reject {analysis['intent_score']} {post['title'][:80]}\")\n        continue\n    kept += 1\n    print(\"=\" * 72)\n    print(f\"{analysis['intent_score']}/10  {analysis['category']}  {analysis['urgency']}\")\n    print(post[\"title\"])\n    print(analysis[\"problem_summary\"])\n    print(\"tools:\", \", \".join(analysis[\"mentioned_tools\"]) or \"(none)\")\n    print(post[\"url\"])\n\nsave_hashes(known)\nprint(f\"kept={kept}  remembered_hashes={len(known)}\")\n```\n\nif **name** == \"**main**\":\n\n    main()\n\n`` ` ``\n\nIf you put this on a schedule, keep the per-run cap, persist hashes, and do not poll faster than the forums themselves update. Be conservative in how you follow up: these are public posts, not a license to spam. Read the thread, and if you reach out, talk about the actual problem in the post.\n\nA useful later upgrade, which I run in production on Sundays, is to look at yield (accepted vs analysed) by source and keyword, and adjust the keyword list and the threshold with guardrails — for example a minimum and maximum number of keywords, and no more than a one-point threshold change at a time. Do not let a model freely rewrite the strategy without those bounds.\n\nI expose the same scored feed as [B2B Buying Intent Signals](https://rapidapi.com/JamesBasset/api/b2b-buying-intent-signals) on RapidAPI. Collection covers Reddit, Hacker News, Lobsters, Dev.to, and Software Recommendations, every six hours. The default list is score 7+. At the time of writing the live feed holds ~100 high-intent B2B signals.\n\n`` `bash ``\n\ncurl --request GET \\\n\n  --url 'https://b2b-buying-intent-signals.p.rapidapi.com/api/v1/signals?tool=shopify&min_score=7&limit=10' \\\n\n  --header 'X-RapidAPI-Host: b2b-buying-intent-signals.p.rapidapi.com' \\\n\n  --header 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY'\n\n`\n\nYou can also filter by `tool` (a case-insensitive exact match on `mentioned_tools`, so `tool=shopify` matches `\"Shopify\"`), `category`, `platform`, and `urgency`. Tools that actually appear in the current feed include Shopify, Supabase, Veeqo, Vesyl, Stripe, Railway, Lovable, Bolt and GitHub — useful starting filters if you sell into ecommerce, payments, or hosting. There is a free BASIC plan (200 calls/month) if you want to inspect real payloads before deciding whether to run any of this yourself.\n\nIf you would rather stay in code, the script above is the whole idea: cheap filters first, a strict JSON contract second, and a threshold that is allowed to say \"no.\"", "url": "https://wpnews.pro/news/how-i-detect-b2b-buying-intent-in-public-developer-forums-with-python-an-llm", "canonical_source": "https://dev.to/jamesbasset/how-i-detect-b2b-buying-intent-in-public-developer-forums-with-python-an-llm-1fo", "published_at": "2026-11-21 17:36:01+00:00", "updated_at": "2026-11-21 17:41:56.514973+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "developer-tools"], "entities": ["Hacker News", "Dev.to", "Stack Exchange", "Reddit", "Lobsters", "Arctic Shift"], "also_reported_by": [], "takeaway": "Pre-filtering forum posts with keyword and software-question rules before sending them to an LLM, then keeping only intent scores of 7 or higher, cuts token spend while surfacing genuine buying-intent posts.", "recommendation": {"advice": "Deduplicate posts by platform plus external ID and pre-filter with cheap keyword rules before calling the LLM, keeping only scores of 7 or higher to cut token spend.", "playbook": "llm-cost", "playbook_url": "https://wpnews.pro/playbooks/llm-cost"}, "alternates": {"html": "https://wpnews.pro/news/how-i-detect-b2b-buying-intent-in-public-developer-forums-with-python-an-llm", "markdown": "https://wpnews.pro/news/how-i-detect-b2b-buying-intent-in-public-developer-forums-with-python-an-llm.md", "text": "https://wpnews.pro/news/how-i-detect-b2b-buying-intent-in-public-developer-forums-with-python-an-llm.txt", "jsonld": "https://wpnews.pro/news/how-i-detect-b2b-buying-intent-in-public-developer-forums-with-python-an-llm.jsonld"}}