Diffing AI Answer Snapshots Over Time With Python

A Hong Kong-based overseas marketing team at PandaWM published a Python approach for tracking how AI assistants describe a brand over time, saving weekly plain-text answer snapshots and diffing them with difflib. The script normalizes whitespace and splits answers into sentences before diffing, flags whether a brand mention was gained or lost, and prints only changed sentences. The team cautions that answers vary between runs, so single-week changes should be treated as hints and paired with referrer and CRM tracking to confirm AI-referred traffic.

If you track how AI assistants describe your brand, you quickly collect a folder of saved answers: one text file per question per week. Reading them all is slow. A small diff script shows what changed, which is usually the only thing worth reviewing. snapshots/ 2026-10-17/q01-best-agency-hk.txt 2026-10-24/q01-best-agency-hk.txt 2026-10-31/q01-best-agency-hk.txt Each file holds the plain-text answer you saved for that question that week, collected manually or through whatever tool your team is allowed to use. python import difflib, pathlib, re, sys BRAND = re.compile r"pandawm|熊猫出海", re.I root = pathlib.Path "snapshots" weeks = sorted p.name for p in root.iterdir if p.is dir prev, curr = weeks -2 , weeks -1 def norm text : text = re.sub r"\s+", " ", text return s.strip for s in re.split r" ?<= . ?。！？ \s ", text if s.strip for f in sorted root / curr .glob " .txt" : old f = root / prev / f.name old = norm old f.read text encoding="utf-8" if old f.exists else new = norm f.read text encoding="utf-8" was, now = any BRAND.search s for s in old , any BRAND.search s for s in new changed = l for l in difflib.unified diff old, new, lineterm="", n=0 if l :1 in "+-" and not l.startswith "+++", "---" flag = "GAINED" if now and not was else "LOST" if was and not now else "" print f"\n {f.stem} {flag} {len changed } changed sentences " for l in changed :12 : print l Splitting into sentences before diffing matters. Assistants rephrase constantly, and a character-level diff would mark everything as changed. Answers vary from run to run even on the same day. Treat one week's change as a hint, and look for patterns across several weeks before acting. Save the date, the tool and the exact prompt with every snapshot so comparisons stay fair. Visibility only matters if it turns into visits and leads. Pair this with referrer tracking and CRM fields so you can see whether a gained mention coincides with more AI-referred traffic; this attributing AI search leads with UTM and CRM https://www.pandawm.com/blog/attribute-google-ai-leads-with-utm-crm.html walks through that setup. Published by PandaWM https://www.pandawm.com/ , an overseas marketing team based in Hong Kong.