How I detect B2B buying intent in public developer forums with Python + an LLM A developer described a Python pipeline for detecting B2B buying intent in public developer and founder forums, fetching recent posts from Hacker News, Dev.to and Stack Exchange Software Recommendations, deduplicating them with SHA-256 hashes, pre-filtering with cheap keyword rules, then scoring each post 1–10 with an LLM that returns strict JSON and keeping only scores of 7 or higher. The production collector runs every six hours with a 36-hour lookback across Reddit, Hacker News, Lobsters, Dev.to and Stack Exchange, using the Arctic Shift archive as the primary Reddit path and official Reddit as fallback to avoid 403 and 429 errors. Keyword alerts are a bad way to find software buyers. Search Reddit or Hacker News for "alternative to" and you will get launch posts, affiliate roundups, venting, and the occasional genuine "I need to replace this before renewal." The last one is the only row I want. Everything else is noise that burns time and, if you score it with a model, tokens. This article walks through the pipeline I use to detect B2B buying intent in public developer and founder forums: fetch recent posts, deduplicate them, pre-filter with cheap text rules, then ask an LLM for a strict JSON score. I run the production collector in TypeScript; the approach is easier to read in Python, which is what I will use here. At the end I will point to a hosted API that already does this, if you would rather not operate a collector yourself. The useful part is the method. You can stop after the script and still have something that works. I am not trying to detect curiosity or brand mentions. I want posts where the author is trying to buy, switch, or replace software: a SaaS tool, an AI product, automation, a developer tool, or a marketing/growth tool. A practical scoring scale, used as an integer from 1 to 10: Two rules matter more than the rest of the prompt: I also ask the model for a category, an urgency low , medium , or high , the software names explicitly cited, and one sentence that describes the buyer's need. The sentence is what you will actually use later, in Slack or in a CRM note. public posts → SHA-256 hash of platform + external id dedup → keyword pre-filter, plus a looser software + question pass → LLM JSON analysis capped per run and per day → keep only if intent score = 7 In production this runs every six hours, looking back 36 hours, against Reddit, Hacker News, Lobsters, Dev.to, and Stack Exchange Software Recommendations. The Python script below uses the three sources that are easiest to call from a laptop: the HN Algolia API, the Dev.to API help and discuss , and the Stack Exchange API for Software Recommendations. The same pre-filter and scorer apply if you add Reddit later. A note on Reddit, because it is the noisiest source and the one people reach for first: official JSON and RSS endpoints often return 403 or 429 from cloud IPs. A production collector needs a polite User-Agent, delays between requests, and a fallback. I use a near-real-time public archive Arctic Shift as the primary Reddit path and official Reddit as a fallback, and I skip any subreddit that errors so one 403 does not kill the run. Stick to public APIs, send a real User-Agent, and sleep between pages. Software Recommendations is a special case: almost every question is "please recommend software," so I treat those posts as buying-intent candidates even when they do not match my keyword list. They still go through the LLM. The pre-filter is skipped; the score threshold is not. Hacker News is searched with quoted intent phrases "alternative to" , "looking to replace" , "willing to pay" , and so on across Ask HN, stories, and comments. Comments are noisy, so I drop any comment that does not also look like a software/tool discussion. Dev.to is simpler: page recent articles tagged help or discuss and keep those inside the time window. python import time import requests USER AGENT = "BuyingIntentTutorial/1.0 dev.to article " HEADERS = {"User-Agent": USER AGENT, "Accept": "application/json"} HN ALGOLIA = "https://hn.algolia.com/api/v1/search by date" INTENT SEARCH PHRASES = '"looking for"', '"alternative to"', '"recommend a"', '"any suggestions"', '"switching from"', '"willing to pay"', '"need a tool"', '"replacement for"', '"looking to replace"', '"frustrated with"', def get json url, params=None, timeout=15 : res = requests.get url, headers=HEADERS, params=params, timeout=timeout res.raise for status return res.json def fetch hn since unix, until unix, max pages=2 : posts, seen = , set for phrase in INTENT SEARCH PHRASES: for tags in "ask hn", "story", "comment" : for page in range max pages : data = get json HN ALGOLIA, { "query": phrase, "tags": tags, "numericFilters": f"created at i ={since unix},created at i<{until unix}", "hitsPerPage": 50, "page": page, }, hits = data.get "hits" or for hit in hits: object id = hit.get "objectID" if not object id or object id in seen: continue title = hit.get "title" or hit.get "story title" or "HN comment" .strip body = hit.get "comment text" or hit.get "story text" or "" if tags == "comment" and not SOFTWARE QUALIFIER.search f"{title}\n{body}" : continue seen.add object id posts.append { "platform": "hackernews", "external id": str object id , "source": "hackernews", "url": f"https://news.ycombinator.com/item?id={object id}", "title": title, "body": strip html body , "author": hit.get "author" , } if not hits or page + 1 = data.get "nbPages", 1 : break time.sleep 0.15 return posts SOFTWARE QUALIFIER and strip html are in the complete script at the end. Dev.to and Software Recommendations follow the same shape: normalize each item into platform , external id , url , title , body , and author . Hash platform and the source's own id, not the title. Titles get edited; ids do not. python import hashlib def source hash post : key = f"{post 'platform' }:{post 'external id' }" return hashlib.sha256 key.encode .hexdigest In production I store two things: accepted signals, and a memory of every post already analysed including rejects . Without the second table, a post that scored 4 yesterday will be sent to the model again tomorrow. That is wasted spend, and it also hides your true yield accepted / analysed , which you need if you ever tune keywords. For a local script, a JSON file of hashes is enough. If you send every Ask HN and every discuss post to an LLM, most of your bill will be spent confirming that a post is not about buying software. A case-insensitive substring check against a short phrase list removes the bulk of that. Phrases that have been useful in practice: alternative to , looking to replace , replacement for , switching from , migrate from looking for , recommend a , any suggestions , any recommendations , can anyone recommend , seeking recommendations , tool recommendations need a tool , need software , need a saas , software for , best tool , what tool do you , how do you automate willing to pay , ready to pay , budget for , in the market for , frustrated with Two exceptions keep this from being too tight: saas , tool , crm , automation , vendor , and similar ? , alternative , recommend , replace , Then rank: more keyword hits first, then longer bodies. Short titles with no context are a poor use of the cap. python def match keywords text, keywords : haystack = " ".join text.lower .split return k for k in keywords if k.lower in haystack def is loose buying candidate title, body : text = f"{title}\n{body}" if not SOFTWARE QUALIFIER.search text : return False return any cue.search text for cue in BUYING QUESTION CUES def to candidates posts, keywords : keyword hits, loose hits = , for post in posts: if post "platform" == "stackexchange" and post "source" == "softwarerecs": keyword hits.append { post, "matched keywords": "softwarerecs" } continue matched = match keywords f"{post 'title' }\n{post 'body' }", keywords if matched: keyword hits.append { post, "matched keywords": matched} elif is loose buying candidate post "title" , post "body" : loose hits.append { post, "matched keywords": "loose intent" } def rank post : return -len post "matched keywords" , -len post "body" keyword hits.sort key=rank loose hits.sort key=rank return keyword hits + loose hits Only then do you slice the queue to a budget MAX AI CALLS per run, plus a daily cap if you put this on a cron . Use an OpenAI-compatible chat API at a low temperature 0.1 . The exact provider does not matter. The contract does: one JSON object, no markdown, no commentary, with a fixed set of keys. Categories I use: ai tools , saas alternative , automation , dev tools , marketing growth The user message is boring on purpose: platform, title, and a body truncated to 2,000 characters. Long threads do not need to be fully scored; the question is usually in the first few paragraphs. CATEGORIES = { "ai tools", "saas alternative", "automation", "dev tools", "marketing growth", } URGENCIES = {"low", "medium", "high"} SYSTEM PROMPT = """ You are a B2B sales-intelligence analyst. You read a public post from Reddit, Hacker News, Lobsters, Dev.to or Stack Exchange Software Recommendations and decide whether its author shows REAL buying intent for software SaaS, AI tools, automation, developer tools, marketing/growth tools : actively searching for a tool, asking for alternatives, frustrated with a current vendor, ready to switch or pay. Scoring guide for intent score integer 1-10 : 1-3 = no purchase intent news, opinion, self-promotion, launch announcement, hiring, rant without need 4-6 = vague interest or early research 7-8 = clear need, actively evaluating solutions 9-10 = urgent, explicit intent to buy/switch now budget, deadline, "need it this week" Posts where the author promotes their OWN product must score <= 3. Respond with ONLY one JSON object, no markdown, no commentary, exactly these keys: { "intent score":