Diff Two Snapshots of a Messy CSV in Python: No-Break Spaces, Duplicate Rows and Renamed Keys

A developer maintaining the VPS Pricing Index published a Python-based approach for diffing successive git snapshots of a hand-maintained CSV, handling thousands separators, currency symbols, footnote daggers, em dashes and no-break spaces that otherwise produce false added/removed rows. The method uses three helpers: a regex-based money parser that returns None for unpublished values, NFKC text normalization to collapse invisible character differences, and a loader that locates the real header below a human-readable title row.

If you build anything on top of a public dataset, sooner or later a new version lands and you have to answer a simple question: what changed? With a hand-maintained CSV, that question is harder than it looks. Keys get renamed, cells carry footnote symbols, and one invisible character can turn an unchanged row into an "added" and a "removed" at the same time. This tutorial builds a small Python diff for two git snapshots of the VPS Pricing Index https://vpshostingradar.com/vps-pricing-index/ , a hosting-price dataset I maintain. I drafted the article with AI assistance; every command was run against the real repository, and the outputs below are copied from those runs and checked against the files by hand. One disclosure: VPS Hosting Radar earns affiliate commissions on some of its pages. This post has no affiliate links. The dataset lives in a public GitHub repository https://github.com/vpshostingradar/vps-pricing-data under CC BY 4.0. pricing.csv has three commits, and git keeps every version: git clone https://github.com/vpshostingradar/vps-pricing-data.git cd vps-pricing-data git log --oneline -- pricing.csv 39a4cad Update pricing.csv 53b7d49 Initial dataset: 16 plans across 13 providers, collected 2026-08-11 8e6fc61 Create pricing.csv git show 53b7d49:pricing.csv aug.csv verified 2026-08-11 git show 39a4cad:pricing.csv sep.csv verified 2026-09-15 git show <commit :<path writes a file exactly as it was at that commit, so you don't need to check anything out. The oldest commit, 8e6fc61 , is a header-only placeholder with a different snake case schema and no data rows, so I left it out. Open aug.csv and you'll see the things that break naive tooling: "2,520.00" with a thousands separator, €4.40‡ with a currency sign and a footnote dagger, € 5.50 with a space, and — an em dash meaning "not published." Cloud VPS 4 and EUR . It looks identical on screen. None of this is unusual. It's what happens when people copy values out of pricing pages and spreadsheets by hand. You want a diff that survives it. Start with three helpers: one that pulls a number out of whatever is in a money cell, one that normalizes text so invisible differences stop mattering, and a loader that finds the real header. python import csv import re import sys import unicodedata from collections import Counter MONEY = re.compile r"-?\d+ ?:\.\d+ ?" def money cell : """'2,520.00' - 2520.0, '€ 5.50' - 5.5, '35.00‡' - 35.0, '—' - None.""" cell = cell or "" .replace ",", "" .strip m = MONEY.search cell return float m.group if m else None def norm text : """NFKC turns U+00A0 no-break space into a plain space; then collapse runs.""" return " ".join unicodedata.normalize "NFKC", text .split def load path : with open path, newline="", encoding="utf-8" as f: lines = list csv.reader f Row 0 is a human-readable title, not the header. Find the real header. start = next i for i, row in enumerate lines if row and row 0 == "Provider" header = lines start return dict zip header, r for r in lines start + 1: if any r A few choices here are deliberate. money returns None for an em dash rather than 0 . "The vendor didn't publish a monthly price" and "the monthly price is zero" are different facts, and the diff should report a move from None to 26.99 as new information, not as a $26.99 price rise. The regex ignores the currency symbol, which is safe only because currency doesn't change within a row between snapshots. If you're diffing a dataset where a row could switch from USD to EUR, keep the symbol as a separate field and compare it too. norm uses NFKC normalization. Among other things, NFKC maps U+00A0 to an ordinary space. str.split with no arguments already treats U+00A0 as whitespace in Python 3, so the split / join pass alone would catch this particular character. NFKC also handles full-width digits and similar compatibility characters, which turn up when values get pasted from some PDFs and from CJK input methods. A diff needs a key. Here the natural key is Provider, Plan . Two things can go wrong with it: duplicates within one file, and keys that change between files because the maintainer swapped one plan for another. python def keyed rows, label : keys = norm r "Provider" , norm r "Plan" for r in rows for key, n in Counter keys .items : if n 1: print f" {label} duplicate key {key} appears {n}x" out = {} for key, r in zip keys, rows : out.setdefault key, r keep first occurrence return out Row replacements are editorial decisions, so they come from the dataset's own notes/changelog, not from fuzzy matching. new key - old key. REPLACES = { "Hostinger", "Web Premium" : "Hostinger", "Cloud Startup" , "Hostinger", "KVM 1" : "Hostinger", "KVM 2" , "InMotion", "VPS 4 vCPU" : "InMotion", "VPS 4 Most Popular " , "ScalaHosting", "Managed Cloud 1" : "ScalaHosting", "Managed Build 1" , } keyed warns about duplicates instead of silently overwriting them. Keeping the first occurrence is fine here because the duplicates are identical; if they weren't, you'd want the script to stop. The REPLACES map is the part people are tempted to automate. Don't. Hostinger has two rows in each snapshot, so "match by provider" is ambiguous. Fuzzy string matching would happily pair KVM 2 with KVM 1 , which happens to be right, but for the wrong reason: that pairing is true because the September notes column says "Replaces KVM 2 8 GB , which overshot the 4 GB baseline," not because the strings are similar. Every entry in the map above comes from a note in the September file or from the repository's CHANGELOG.md . When the next snapshot arrives, you read the changelog and update the map. It takes two minutes and you'll trust the output afterwards. FIELDS = { "First term $/mo": "intro", "Prepay months": "prepay", "Monthly $/mo": "monthly", "Renewal $/mo": "renewal", "36-month total $": "total 36", } def compare old, new : matched = set for key, n in new.items : okey = key if key in old else REPLACES.get key if okey not in old: print f"+ added {key 0 } {key 1 }" continue matched.add okey tag = "same plan" if okey == key else f"replaces {okey 1 }" changes = for col, short in FIELDS.items : a, b = money old okey .get col , money n.get col if a = b: changes.append f"{short} {a} - {b}" if changes: print f"~ changed {key 0 } {key 1 } {tag} : " + "; ".join changes for okey in old.keys - matched: print f"- removed {okey 0 } {okey 1 }" if name == " main ": old = keyed load sys.argv 1 , "old" new = keyed load sys.argv 2 , "new" print f"old: {len old } unique plans, new: {len new } unique plans\n" compare old, new Save the three blocks as snapdiff.py and run python3 snapdiff.py aug.csv sep.csv . Here's the output, with one changed line removed for length marked … : old duplicate key 'Liquid Web', 'Managed VPS 4 GB' appears 2x old duplicate key 'Contabo', 'Cloud VPS 4 EUR ' appears 2x old: 16 unique plans, new: 19 unique plans ~ changed SiteGround StartUp same plan : monthly 24.99 - 17.99 ~ changed Hostinger Web Premium replaces Cloud Startup : intro 7.99 - 2.99; prepay 48.0 - 12.0; monthly 27.99 - 11.99; renewal 25.99 - 11.99; total 36 287.64 - 323.64 ~ changed Hostinger KVM 1 replaces KVM 2 : intro 8.79 - 6.49; monthly 24.49 - 19.49; renewal 14.99 - 19.49; total 36 390.84 - 389.64 ~ changed InMotion VPS 4 vCPU replaces VPS 4 Most Popular : intro 10.99 - 13.99; monthly None - 26.99; total 36 587.64 - 659.64 ~ changed ScalaHosting Managed Cloud 1 replaces Managed Build 1 : prepay 36.0 - 1.0; total 36 1078.2 - 1953.2 + added ScalaHosting Unmanaged Config 1 ~ changed WP Engine Startup same plan : intro 30.0 - 28.0; total 36 1200.0 - 1176.0 ~ changed Contabo Cloud VPS 4 EUR same plan : intro 4.4 - 5.28; monthly None - 6.6; renewal 5.5 - 6.6; total 36 None - 205.92 … + added SiteGround Cloud Jump Start 4 CPU / 8 GB / 40 GB + added Cloudways Autonomous Growth The omitted line is the Rocket Starter row, where prepay goes from 1 to 12. Sixteen unique plans become nineteen, with three real additions and nothing removed. That matches the changelog, which lists exactly those three as new rows. To see what normalization buys, replace the body of norm with return text.strip and run it again. The added / removed lines become: + added ScalaHosting Unmanaged Config 1 + added Contabo Cloud VPS 4 EUR + added Cloudways DigitalOcean 4 GB Basic premium CPU + added SiteGround Cloud Jump Start 4 CPU / 8 GB / 40 GB + added Cloudways Autonomous Growth - removed Contabo Cloud VPS 4 EUR - removed Cloudways DigitalOcean 4 GB Basic premium CPU That's seven lines where the true answer is three. Two unchanged plans show up as churn because their August names contain a no-break space. The printed names look identical in the terminal, which is exactly what makes this bug hard to spot. Drop the REPLACES map as well and four more plans would appear as removed-plus-added pairs, leaving you with a report that's mostly noise. Here's the part I think matters most. The script reports that values changed, but it can't tell a vendor price change from a correction in the dataset, and in this pair of snapshots most of the movement is corrections. monthly 24.99 - 17.99 intro 10.99 - 13.99 The rest WP Engine's intro rate, ScalaHosting's prepay term, the Rocket prepay term aren't explained line by line in the changelog. For those, the diff gives you a short list of rows to check against each vendor's page, and the vendor page is the tiebreaker. That's a good reason to keep editorial context, meaning notes and a changelog, next to any hand-collected dataset. A diff turns two hundred cells into ten lines worth reading. A human still has to say what each line means. I wrote about the collection process and the mistakes that shaped it https://medium.com/@vpshostingradar/how-i-hand-collected-pricing-for-16-hosting-plans-and-the-mistake-that-almost-broke-the-dataset-7e67e008e06b on Medium, if you want the background. The script is short enough to adapt rather than configure: keyed to whatever uniquely identifies a row in your data. If there's an ID column, use it. The current site export at /data/vps-pricing-index.csv has a stable id column hostinger-kvm-1 , siteground-startup and so on , which removes the whitespace problem entirely. You'd still map deliberately replaced plans by hand. REPLACES small and sourced. print in The site's tools page https://vpshostingradar.com/tools/ lists the CSV and JSON exports if you want the current snapshot instead of the git history. Prices in both are dated the latest verification was 2026-09-15 , so check the vendor's page before you act on any of them. Carter Wang is a serial founder who has bought hosting and run production stacks for more than a decade. He maintains the VPS Pricing Index at VPS Hosting Radar https://vpshostingradar.com/ .