Cursor vs Copilot Survey Reweighting: 12 Steps, 70 Min A tutorial-style build called survey_rake shows that Cursor-versus-GitHub Copilot rankings built on developer survey data shift substantially depending on the target team composition, using raking to reweight a synthetic file of 4,000 invented respondents against three target distributions (language, organization size, experience) in roughly 200 lines of Python and about 70 minutes. The piece grounds the problem in real survey figures: Stack Overflow's 2024 edition collected 65,437 responses from 185 countries in May 2024, reporting 62% of respondents currently using AI tools in development (up from 44% in 2023) and 76% using or planning to use them, while 74% of respondents used Visual Studio Code and JetBrains' research hub cites more than 20,000 developers a year across 194 countries. The author stresses that none of the outputs are real Cursor or Copilot market share, since every such dataset describes a self-selected crowd rather than a random sample of professional developers. A survey percentage for Cursor or GitHub Copilot tells you how many of the people who answered said they use the tool. It does not tell you how many developers use it worldwide, and it says even less about your own team. Stack Overflow’s 2024 edition collected 65,437 responses from 185 countries in May 2024, and the 2026 results are already circulating, with TechRepublic’s report on the 2026 survey https://www.techrepublic.com/article/news-stack-overflow-2026-javascript-rust-python/ leading on programming languages rather than AI assistants. Every one of those datasets describes a self-selected crowd. That gap between who answered and who exists sits under every Cursor vs Copilot comparison built on survey data. A sample packed with solo JavaScript developers will flatter an editor-first tool. A sample packed with Java engineers at large companies will flatter the incumbent that already lives inside their IDE. Raking, a standard survey-weighting method, closes the composition gap, and it takes about 200 lines of Python. If you want counting and charts before any weighting, the pandas walkthrough of Stack Overflow survey data https://shattered.io/cursor-vs-copilot-stack-overflow-survey-pandas-12-steps-2026/ is the gentler on-ramp. The build is a small package named survey rake. It loads a respondent file, reshapes the sample to match three target distributions language, organization size, experience , reports weighted shares with bootstrap intervals, and compares three team scenarios side by side. Every output below comes from a synthetic file of 4,000 invented respondents that the code generates itself. Nothing in those outputs is a real Cursor or Copilot market share, and the one result worth remembering is how far the ranking moves when the target team changes. Don't miss new tech stories on Google Add Shattered once in the Google app and our stories show up in your news suggestions. Add now https://www.google.com/preferences/source?hl=en&q=https%3A%2F%2Fshattered.io%2F Why Survey Shares Mislead Any Cursor vs Copilot Comparison Start with what the big surveys actually measure. The Stack Overflow 2024 survey reported that 62% of respondents were currently using AI tools in development, up from 44% in 2023, and that 76% were using or planning to use them. Those are two different questions with two different answers, and the gap between them is 14 points. In the same survey, 41% of ChatGPT users said they wanted to use GitHub Copilot the following year. That figure measures desire among one subgroup, not usage across the field, and a careless headline can still turn it into a market share claim in one edit. JetBrains publishes a second major source. Its research hub describes surveys built on responses from more than 20,000 developers a year across 194 countries since 2015, and it lists a State of Developer Ecosystem report for 2026 https://www.jetbrains.com/developer-research/ alongside every edition back to 2017. The 2026 edition ran past 500 questions, up from about 150 in the first survey. More questions means more definitions, and more definitions means more ways for two numbers that look comparable to measure different things. The deeper problem is selection. Nobody drew these respondents at random from all professional developers. People who answer a developer survey skew toward those who read developer sites, use certain editors, and care about tooling. A survey can tell you that, among its respondents, the share who named a given assistant was some number. It cannot tell you the share of the whole field without further assumptions. Treat any claim that a survey proves market share with suspicion. The Stack Overflow 2024 pages themselves show that 74% of respondents used Visual Studio Code, which is a useful reminder that the sample leans toward one editor ecosystem long before AI tools enter the question. What you can do is ask a narrower and more honest question. If your team has a known mix of languages, company sizes and seniority levels, you can reweight the survey so its respondents look like your team on those traits, then read the Cursor and Copilot shares for that reweighted crowd. That does not cure self-selection, and it does not turn a survey into a census. It does stop a sample dominated by one kind of developer from answering a question about a different kind. For the broader audit habits around these numbers, the market share audit for Cursor and GitHub Copilot https://shattered.io/cursor-vs-github-copilot-market-share-audit-12-steps-2026/ covers how to trace each published percentage back to its source. Raking suits this job for three reasons. It needs only the marginal distributions of your team three short tables, not a full cross-tabulation , it converges in a handful of passes on well-behaved data, and it produces one weight per respondent that you can reuse for any question in the same survey. The cost is variance. Respondents from underrepresented groups carry large weights, so the effective sample size shrinks, and that shrinkage is the number most write-ups hide. Prerequisites and Tested Versions The project needs Python, a virtual environment and three libraries. Everything ran on the versions in the table below. The code uses only standard pandas and NumPy calls, so nearby versions should behave the same, but every printed output here came from exactly this stack. | Component | Version used | Role | |---|---|---| | Python | 3.12.3 | Runs the package and the CLI | | pandas | 3.1.1 | Loads the CSV and handles the multi-select column | | NumPy | 2.5.5 | Weights, bincount marginals, bootstrap resampling | | pytest | 9.2.0 | Guards the raking math with five tests | | Operating system | Linux any shell with venv support | Commands use a POSIX shell | You also need basic comfort with pandas DataFrames, a terminal, and about 70 minutes. No survey-statistics background is required, because each step explains the one idea it introduces. You do not need to download any real survey file to follow along. The synthetic generator produces a file with the same shape as a typical multi-select survey export, and a later step shows how to point the same code at a real download. If you do use a real file, read the license first. Stack Overflow’s survey pages state that the data is licensed under ODbL 1.0 and the contents under DbCL 1.0. Those terms carry attribution and sharing conditions that apply to derived databases, so record them in your ledger Step 3 and check them before you publish weighted extracts. Treat the synthetic file as the default and the real file as an optional upgrade once the pipeline passes its tests. Plan for the time like this. Environment and project setup take about 10 minutes. Writing the loader and raking code takes 25. Diagnostics, bootstrap and scenarios take 25. The final 10 minutes go to labeling and the pitfalls checklist. The bootstrap is the only slow part, and a 500-replicate run per scenario finished in under 10 seconds on the machine used here. Steps 1-3: Define the Metric, Set Up the Project, Start a Survey Ledger Step 1: Define the metric before you touch data. Write one sentence naming exactly what you will estimate. Survey results blur at least five different things, and the numbers rarely line up across surveys. The table lists the common ones, with examples drawn from the surveys described above. | Metric | Typical question shape | What it can support | Example from the sources | |---|---|---|---| | Awareness | Have you heard of this tool? | Reach of a brand | Not a usage figure | | Ever used | Have you ever used this tool? | Trial, not retention | Rarely published for coding assistants | | Currently uses | Which tools do you use now? | Current adoption among respondents | 62% of 2024 Stack Overflow respondents used AI tools in development | | Plans to use | Do you use or plan to use? | Intent plus usage | 76% in the same survey | | Wants next year | Which tools do you want to use? | Desire in a subgroup | 41% of ChatGPT users wanted GitHub Copilot | | Paid seat share | Which plan do you pay for? | Revenue-side adoption | Needs vendor or billing data | The pipeline estimates one metric: the share of answered respondents who say they currently use each tool at work. Keep the sentence somewhere you will see it again, because it becomes the basis label in the final report. A share claim linter written in Python https://shattered.io/cursor-vs-copilot-share-claim-linter-python-12-steps-2026/ automates the same discipline for published claims, flagging sentences that blur these categories. Step 2: Set up the project. Create a folder, a virtual environment and the package skeleton. Run the commands from the project root, because every later command uses python -m so that the package imports resolve without any install step. mkdir survey-rake-lab && cd survey-rake-lab python3 -m venv .venv source .venv/bin/activate pip install pandas numpy pytest mkdir -p survey rake/tests data touch survey rake/ init .py The finished layout has eight Python files and three data files. Keep it exactly like this so the commands in later steps work unchanged. data/ledger.json data/synthetic survey.csv data/targets.json survey rake/ init .py survey rake/cli.py survey rake/estimate.py survey rake/loader.py survey rake/rake.py survey rake/report.py survey rake/synth.py survey rake/tests/test rake.py Step 3: Start a survey ledger. Before loading a single row, write down the survey’s metadata in data/ledger.json . Fieldwork dates, respondent counts, question wording and license terms belong next to the data, not in your memory. Two entries are enough to start. The first records the Stack Overflow 2024 facts quoted earlier. The second shows how to leave honest placeholders for a survey you have not finished reading. { "surveys": { "id": "so-2024", "publisher": "Stack Overflow", "edition": 2024, "fieldwork": "May 2024", "respondents": 65437, "countries": 185, "license": "ODbL 1.0 for the data, DbCL 1.0 for the contents", "metric": "currently uses AI tools in development multi-select ", "recruitment": "TODO: copy from the methodology page", "weights shipped": "TODO: check the schema file" }, { "id": "jb-2026", "publisher": "JetBrains", "edition": 2026, "fieldwork": "TODO: copy from the report", "respondents": null, "countries": null, "license": "TODO: check the download page", "metric": "TODO: copy the exact question wording", "recruitment": "TODO: copy from the methodology page", "weights shipped": "TODO: check the schema file" } } The placeholders are deliberate. A field that reads TODO is a visible gap, while a field filled from memory is a hidden error. The ledger also solves a date problem that bites every survey article. A survey page can show a November 2026 update stamp while its responses were collected in May 2024, so the page date and the fieldwork date are different facts and must live in different fields. Steps 4-5: Build the Respondent Table Step 4: Generate synthetic respondents. The generator invents 4,000 developers with three traits language, organization size, experience and a semicolon-delimited column of AI tools they use at work. It plants realistic structure on purpose. Small and solo shops lean toward Cursor, large organizations lean toward GitHub Copilot, and Java developers lean slightly further toward Copilot. The sample itself skews toward solo and small-company developers and toward JavaScript and Python. That mismatch is exactly what raking has to repair. python """Synthetic respondents for testing. Not real survey data.""" import numpy as np import pandas as pd LANGS = "Python", "JavaScript", "Java", "Other" ORGS = "solo", "small", "mid", "large" EXPS = "0-4", "5-14", "15+" def make sample n: int = 4000, seed: int = 2026 - pd.DataFrame: rng = np.random.default rng seed lang = rng.choice LANGS, n, p= 0.30, 0.35, 0.12, 0.23 org = rng.choice ORGS, n, p= 0.30, 0.35, 0.20, 0.15 exp = rng.choice EXPS, n, p= 0.30, 0.45, 0.25 def logit x : return 1 / 1 + np.exp -x org cursor = {"solo": 0.6, "small": 0.4, "mid": -0.1, "large": -0.7} org copilot = {"solo": -0.9, "small": -0.4, "mid": 0.3, "large": 0.9} lang copilot = {"Java": 0.5, "Python": 0.0, "JavaScript": -0.1, "Other": -0.1} exp cursor = {"0-4": 0.3, "5-14": 0.0, "15+": -0.3} p cursor = logit -1.2 + np.array org cursor o for o in org + np.array exp cursor e for e in exp p copilot = logit -0.6 + np.array org copilot o for o in org + np.array lang copilot l for l in lang p chat = logit 0.9 + 0 p cursor rows = for i in range n : tools = if rng.random < p chat i : tools.append "ChatGPT" if rng.random < p copilot i : tools.append "GitHub Copilot" if rng.random < p cursor i : tools.append "Cursor" if rng.random < 0.12: tools.append "Claude Code" rng.shuffle tools rows.append ";".join tools if tools else " none " return pd.DataFrame { "respondent id": np.arange 1, n + 1 , "language": lang, "org size": org, "experience": exp, "ai tools work": rows, } if name == " main ": import sys out = sys.argv 1 if len sys.argv 1 else "data/synthetic survey.csv" make sample .to csv out, index=False print f"wrote {out}" Generate the file and look at the first rows. Note that respondent 1 answered with the literal text none . The generator writes that marker on purpose, and the pitfalls section explains why a blank cell and an explicit none answer must never be confused. python -m survey rake.synth data/synthetic survey.csv head -4 data/synthetic survey.csv wrote data/synthetic survey.csv respondent id,language,org size,experience,ai tools work 1,Python,large,5-14, none 2,JavaScript,small,0-4,ChatGPT 3,JavaScript,large,0-4,ChatGPT To use a real survey export instead, open the schema or codebook that ships with the download and find the column that lists the AI tools each respondent currently uses. Most exports store a multi-select answer as one text cell with a delimiter. Pass that column name to the CLI with --tool-col and recode your three trait columns into the level names used in your targets file. Do not guess column names. Read the schema file, because survey editions rename and restructure columns between years. Step 5: Load and normalize the multi-select answer. The loader turns the delimited cell into one boolean per tool. It matches names case-insensitively after trimming whitespace, and it keeps an alias list because respondents and survey designers write the same product several ways. The GitHub Copilot entry accepts both the full name and the short form. By default it drops respondents who skipped the question, so every later percentage uses answered respondents as its denominator. python import pandas as pd ALIASES = { "Cursor": "Cursor" , "GitHub Copilot": "GitHub Copilot", "Copilot" , } def load survey path: str, tool col: str, sep: str = ";", drop unanswered: bool = True - pd.DataFrame: df = pd.read csv path if tool col not in df.columns: raise KeyError f"{tool col r} not in columns: {list df.columns }" answered = df tool col .notna df = df.assign answered=answered for tool, names in ALIASES.items : wanted = {n.strip .lower for n in names} flag = df tool col .fillna "" .map lambda cell: bool wanted & {t.strip .lower for t in cell.split sep } df f"uses {tool.lower .replace ' ', ' ' }" = flag & answered if drop unanswered: df = df df "answered" .reset index drop=True return df Check the raw numbers and the trait mix before any weighting. These are the unweighted baselines that every later step has to improve on. python python - <<'EOF' from survey rake.loader import load survey df = load survey "data/synthetic survey.csv", "ai tools work" print len df , df.answered.mean print df "uses cursor", "uses github copilot" .mean .round 3 print df.language.value counts normalize=True .round 3 .to dict print df.org size.value counts normalize=True .round 3 .to dict print df.experience.value counts normalize=True .round 3 .to dict EOF 4000 1.0 uses cursor 0.276 uses github copilot 0.335 dtype: float64 {'JavaScript': 0.36, 'Python': 0.294, 'Other': 0.226, 'Java': 0.12} {'small': 0.343, 'solo': 0.298, 'mid': 0.202, 'large': 0.156} {'5-14': 0.446, '0-4': 0.296, '15+': 0.258} In the raw sample, Copilot shows a higher usage share than Cursor. Large organizations and Java developers make up only a small part of respondents, which is the imbalance the next steps correct. A reader who stopped here would conclude that Copilot leads Cursor among all developers. Hold that conclusion loosely, because the answer depends on whose team you ask about. Step 6: Write the Target Mix File A target file states what your population looks like on each weighting variable. Each variable is a list of levels with shares that sum to one. The file also carries a metric label and a source note per scenario, and the CLI prints that label with every result so the basis travels with the numbers. Use three scenarios. The startup scenario describes a small-company, JavaScript-and-Python shop. The enterprise scenario describes a large-company team with a heavy Java presence. The balanced scenario sits close to the raw sample, which makes it a useful control, because weighting a sample toward its own mix should change very little. { "metric label": "currently uses at work multi-select answer, synthetic data ", "scenarios": { "startup": { "source": "our HR export, Nov 2026", "marginals": { "language": { "Python": 0.45, "JavaScript": 0.4, "Java": 0.05, "Other": 0.1 }, "org size": { "solo": 0.1, "small": 0.7, "mid": 0.15, "large": 0.05 }, "experience": { "0-4": 0.4, "5-14": 0.45, "15+": 0.15 } } }, "enterprise": { "source": "our HR export, Nov 2026", "marginals": { "language": { "Python": 0.25, "JavaScript": 0.2, "Java": 0.4, "Other": 0.15 }, "org size": { "solo": 0.02, "small": 0.05, "mid": 0.25, "large": 0.68 }, "experience": { "0-4": 0.2, "5-14": 0.5, "15+": 0.3 } } }, "balanced": { "source": "our HR export, Nov 2026", "marginals": { "language": { "Python": 0.35, "JavaScript": 0.3, "Java": 0.15, "Other": 0.2 }, "org size": { "solo": 0.2, "small": 0.3, "mid": 0.25, "large": 0.25 }, "experience": { "0-4": 0.3, "5-14": 0.45, "15+": 0.25 } } } } } Every share in that file is a placeholder for your own data. Replace each one with numbers from a source you can cite, such as an HR export, a repository language census or a headcount report, and put the source and date in the source field. Do not take targets from a survey and then weight that same survey toward them. That circular move launders the sample’s bias into your estimate. Three rules keep targets usable. First, every level present in the data needs an entry in the target, even at a tiny share, because the code refuses to run when it finds a level with no target. Second, every level with a positive target needs at least one respondent, because no weight can create a person who does not exist. Third, avoid exact zeros. The enterprise scenario gives solo developers 2% instead of 0%, since a zero target pushes weights toward zero while the lower trim bound holds them up, and the two forces never settle. Choose variables that plausibly relate to tool choice and that you can measure on your own team. Language, company size and experience qualify. Variables that you cannot measure at home, such as the respondent’s favorite editor theme, do not. Each added variable raises the number of constraints, which raises the chance of non-convergence and widens the weight spread, so three variables is a sensible ceiling for a first pass. Step 7: Implement Raking With Iterative Proportional Fitting Raking adjusts weights one variable at a time. Start every respondent at weight 1. Take the first variable, compute the weighted share of each level, and multiply each respondent’s weight by the ratio of target share to current share for their level. Move to the next variable and repeat. After the last variable, the first variable’s shares have drifted slightly off target, so loop again. Each full pass shrinks the largest gap between weighted and target shares, and the loop stops when that gap falls below a tolerance. The code below follows that recipe in vectorized NumPy. It first encodes each variable to integer codes and validates the targets, raising a RakeError for the three failure modes from Step 6. Inside the loop, np.bincount with weights computes each level’s weighted total in one call. After each pass, it clips weights to a band around the mean trimming and measures the worst remaining gap. The function returns the weights, rescaled to mean 1, plus a diagnostics dictionary with the iteration count, the final gap, a convergence flag and the gap history. python import numpy as np import pandas as pd class RakeError ValueError : pass def encode df: pd.DataFrame, targets: dict : codes, shares = {}, {} for col, mix in targets.items : if col not in df.columns: raise RakeError f"target column {col r} missing from data" total = sum mix.values if abs total - 1.0 1e-6: raise RakeError f"{col}: target shares sum to {total:.4f}, not 1" levels = list mix seen = set df col .dropna .unique extra = seen - set levels if extra: raise RakeError f"{col}: data has levels with no target: {sorted extra }" empty = l for l in levels if mix l 0 and l not in seen if empty: raise RakeError f"{col}: target levels with no respondents: {empty}" codes col = df col .map {l: i for i, l in enumerate levels } .to numpy if np.isnan codes col .astype float .any : raise RakeError f"{col}: missing values, drop or impute first" codes col = codes col .astype int shares col = np.array mix l for l in levels return codes, shares def rake df, targets, max iter=200, tol=1e-6, trim= 0.2, 5.0 , codes shares=None : codes, shares = codes shares or encode df, targets n = len df w = np.ones n gap = np.inf history = for it in range 1, max iter + 1 : for col, c in codes.items : tot = np.bincount c, weights=w, minlength=len shares col factor = np.where tot 0, shares col w.sum / np.maximum tot, 1e-12 , 1.0 w = w factor c if trim: m = w.mean w = np.clip w, trim 0 m, trim 1 m gap = max np.abs np.bincount c, weights=w, minlength=len shares col / w.sum - shares col .max for col, c in codes.items history.append gap if gap < tol: break w = w / w.mean return w, {"iterations": it, "max gap": float gap , "converged": bool gap < tol , "history": history} The trim parameter deserves attention, because it is the one dial that trades bias against variance. Trimming caps extreme weights so that a few respondents cannot dominate an estimate. The default band lets a weight fall to 0.2 times the mean or rise to 5 times the mean. Tighter bands give steadier estimates, but they may prevent the weighted margins from reaching the targets at all. The next step shows exactly that failure and how the diagnostics expose it. A word on the algorithm’s limits. Raking matches margins, not joint distributions. If your team has many large-company Java developers in particular but the sample has almost none, matching the language margin and the size margin separately does not guarantee the Java-at-large-company cell is right. When you know the joint table, rake on the combined cell instead see Advanced Tips . When you only know margins, which is the common case, raking is the standard tool. Step 8: Read the Diagnostics Before the Percentages Never look at a weighted share until you have looked at the weights. Run the startup targets and print the convergence history, the weighted margin for organization size and a summary of the weight distribution. python python - <<'EOF' import json, pandas as pd from survey rake.loader import load survey from survey rake.rake import rake df = load survey "data/synthetic survey.csv", "ai tools work" t = json.load open "data/targets.json" "scenarios" "startup" "marginals" w, info = rake df, t print f"{h:.4f}" for h in info "history" print pd.Series w .groupby df "org size" .sum / w.sum .round 3 .to dict print pd.Series w .describe .round 2 .to dict EOF '0.0086', '0.0026', '0.0010', '0.0004', '0.0002', '0.0001', '0.0000', '0.0000', '0.0000', '0.0000', '0.0000' {'large': 0.05, 'mid': 0.15, 'small': 0.7, 'solo': 0.1} {'count': 4000.0, 'mean': 1.0, 'std': 1.06, 'min': 0.2, '25%': 0.27, '50%': 0.48, '75%': 1.22, 'max': 4.56} Read the three lines in order. The gap history falls to effectively zero share error after several passes, so the algorithm converged. The weighted organization-size margin lands exactly on the startup target of 10% solo, 70% small, 15% mid and 5% large. The weight summary is the warning light. Weights spread widely, and the median weight sits well below the mean. A median far below the mean means most respondents are downweighted while a minority carries the correction, which is what happens when the sample is heavy on solo developers and the target is heavy on small companies. Now lock the behavior in with tests. Five short tests cover the properties that matter. Weighted margins must match targets after raking. An over-tight trim must report non-convergence instead of pretending success. Unknown levels and bad share sums must raise errors. Equal weights must give an effective sample size equal to the head count. python import numpy as np import pandas as pd import pytest from survey rake.rake import rake, RakeError from survey rake.estimate import kish neff from survey rake.synth import make sample TARGETS = { "language": {"Python": 0.4, "JavaScript": 0.2, "Java": 0.2, "Other": 0.2}, "org size": {"solo": 0.1, "small": 0.2, "mid": 0.3, "large": 0.4}, } def test margins match : df = make sample 3000 w, info = rake df, TARGETS assert info "converged" got = pd.Series w .groupby df "language" .sum / w.sum for k, v in TARGETS "language" .items : assert got k == pytest.approx v, abs=1e-4 def test tight trim flags non convergence : df = make sample 3000 w, info = rake df, TARGETS, trim= 0.5, 2.0 assert not info "converged" assert info "max gap" 0.01 def test unknown level raises : df = make sample 500 bad = {"language": {"Python": 0.5, "Java": 0.5}} with pytest.raises RakeError : rake df, bad def test shares must sum to one : df = make sample 500 with pytest.raises RakeError : rake df, {"language": {"Python": 0.5, "JavaScript": 0.2, "Java": 0.2, "Other": 0.2}} def test neff is n for equal weights : assert kish neff np.ones 100 == pytest.approx 100 python -m pytest -v survey rake/tests survey rake/tests/test rake.py::test margins match PASSED 20% survey rake/tests/test rake.py::test tight trim flags non convergence PASSED 40% survey rake/tests/test rake.py::test unknown level raises PASSED 60% survey rake/tests/test rake.py::test shares must sum to one PASSED 80% survey rake/tests/test rake.py::test neff is n for equal weights PASSED 100% ============================== 5 passed in 0.39s =============================== The tight-trim test records a real finding from this dataset. With a band of 0.5 to 2.0 and a simple two-variable target, the algorithm still had a maximum gap of about 7 percentage points after 200 iterations. That is why the function returns a convergence flag rather than only weights. A report built on non-converged weights looks fine, prints plausible numbers and quietly misses its own targets. Steps 9-10: Weighted Shares, Effective Sample Size and Bootstrap Intervals Step 9: Compute weighted shares and the effective sample size. A weighted share is the sum of weights over respondents who use the tool, divided by the sum of all weights. The effective sample size, from the Kish approximation, is the squared sum of weights divided by the sum of squared weights. It answers a plain question: how many equally weighted respondents would give the same precision? The design effect is the head count divided by that number. A design effect of 2 means the weighting cost you half your sample. python import numpy as np import pandas as pd from .rake import rake, encode def kish neff w - float: w = np.asarray w, dtype=float return float w.sum 2 / w 2 .sum def weighted share flag, w - float: flag = np.asarray flag, dtype=float return float flag w .sum / w.sum These imports and two functions open survey rake/estimate.py , and the bootstrap below follows in the same file. A textbook formula for a confidence interval on a plain proportion does not apply to weighted data, which is why an unweighted Wilson interval for the Cursor vs Copilot gap https://shattered.io/cursor-vs-copilot-margin-of-error-wilson-interval-12-steps-2026/ understates the uncertainty here. The weights add variance that the simple formula never sees. Step 10: Bootstrap with re-raking. The bootstrap resamples respondents with replacement, rakes each resample from scratch, computes the share and repeats several hundred times. The 2.5th and 97.5th percentiles of those shares form a 95% interval. Re-raking inside every replicate is the critical detail. If you rake once and then resample the fixed weights, you ignore the variance created by the weighting itself, and the intervals come out too narrow. python def bootstrap ci df, flag col, targets, reps=500, seed=7, trim= 0.2, 5.0 , alpha=0.05 : rng = np.random.default rng seed n = len df est = failed = 0 for in range reps : idx = rng.integers 0, n, n sample = df.iloc idx .reset index drop=True try: w, info = rake sample, targets, trim=trim except ValueError: failed += 1 continue if not info "converged" : failed += 1 continue est.append weighted share sample flag col , w if not est: raise RuntimeError "no converged replicates: loosen the trim or coarsen the targets" lo, hi = np.quantile est, alpha / 2, 1 - alpha / 2 return float lo , float hi , failed Two safety details sit in that function. A resample can, by chance, miss a rare level entirely, and the raking validation then raises an error that the loop catches and counts. A resample can also fail to converge. Both cases increment a failure counter that the report prints, so you can see whether the intervals rest on 500 replicates or on 300. If every replicate fails, the function raises a clear error instead of returning an empty interval. The seed is fixed at 7, so the printed numbers reproduce exactly. In real work, run the bootstrap with two or three different seeds once and confirm the interval endpoints move by less than a tenth of a percentage point. If they move more, raise the replicate count before trusting the third significant figure of anything. Step 11: Run Three Scenarios and Read the Sensitivity Table The report module ties the pieces together. For each requested tool flag it computes the raw share, the weighted share and the bootstrap interval, then attaches the diagnostics. It refuses to continue when the main raking pass did not converge, and the CLI wraps it with trim flags and an all-scenarios mode. python import pandas as pd from .estimate import bootstrap ci, kish neff, weighted share from .rake import RakeError, rake def build report df, targets, flags, reps=500, trim= 0.2, 5.0 : w, info = rake df, targets, trim=trim if not info "converged" : raise RakeError f"raking stopped at max gap {info 'max gap' :.4f} after {info 'iterations' } iterations: loosen trim={trim}" rows = for flag in flags: lo, hi, failed = bootstrap ci df, flag, targets, reps=reps, trim=trim rows.append { "metric": flag.removeprefix "uses " , "raw pct": round 100 df flag .mean , 1 , "weighted pct": round 100 weighted share df flag , w , 1 , "ci95 low": round 100 lo, 1 , "ci95 high": round 100 hi, 1 , "failed reps": failed, } diag = { "n": len df , "n eff": round kish neff w , 1 , "design effect": round len df / kish neff w , 2 , "w min": round float w.min , 2 , "w max": round float w.max , 2 , {k: info k for k in "iterations", "converged" }, } return pd.DataFrame rows , diag python import argparse, json, sys from .loader import load survey from .rake import RakeError from .report import build report def run df, cfg, name, reps, trim : targets = cfg "scenarios" name "marginals" print f"scenario: {name} trim={trim}" print f"basis: {cfg 'metric label' }" try: table, diag = build report df, targets, "uses cursor", "uses github copilot" , reps=reps, trim=trim except RakeError as e: print f"RAKE FAILED: {e}" return False print table.to string index=False print diag return True def main : ap = argparse.ArgumentParser ap.add argument "--csv", required=True ap.add argument "--tool-col", required=True ap.add argument "--targets", required=True ap.add argument "--scenario", required=True ap.add argument "--reps", type=int, default=500 ap.add argument "--trim-lo", type=float, default=0.2 ap.add argument "--trim-hi", type=float, default=5.0 a = ap.parse args cfg = json.load open a.targets df = load survey a.csv, a.tool col names = list cfg "scenarios" if a.scenario == "all" else a.scenario ok = run df, cfg, n, a.reps, a.trim lo, a.trim hi for n in names sys.exit 0 if all ok else 2 if name == " main ": main Run every scenario with the default trim. Two scenarios succeed and one fails, and the failure is informative. python -m survey rake.cli --csv data/synthetic survey.csv --tool-col ai tools work \ --targets data/targets.json --scenario all scenario: startup trim= 0.2, 5.0 basis: currently uses at work multi-select answer, synthetic data metric raw pct weighted pct ci95 low ci95 high failed reps cursor 27.6 30.1 28.0 32.3 0 github copilot 33.5 31.1 29.0 33.0 0 {'n': 4000, 'n eff': 1880.6, 'design effect': 2.13, 'w min': 0.2, 'w max': 4.56, 'iterations': 11, 'converged': True} scenario: enterprise trim= 0.2, 5.0 basis: currently uses at work multi-select answer, synthetic data RAKE FAILED: raking stopped at max gap 0.1365 after 200 iterations: loosen trim= 0.2, 5.0 scenario: balanced trim= 0.2, 5.0 basis: currently uses at work multi-select answer, synthetic data metric raw pct weighted pct ci95 low ci95 high failed reps cursor 27.6 24.8 23.5 26.2 0 github copilot 33.5 38.5 36.9 40.0 0 {'n': 4000, 'n eff': 3511.3, 'design effect': 1.14, 'w min': 0.54, 'w max': 2.04, 'iterations': 3, 'converged': True} The enterprise scenario asks a sample with few large-company and Java respondents to impersonate a team that is mostly large-company and heavily Java. Capped weights cannot stretch that far, so the run stops with a wide gap and the CLI exits with a nonzero status. Loosen the band deliberately and rerun that scenario, accepting the price in variance. python -m survey rake.cli --csv data/synthetic survey.csv --tool-col ai tools work \ --targets data/targets.json --scenario enterprise --trim-lo 0.05 --trim-hi 15 scenario: enterprise trim= 0.05, 15.0 basis: currently uses at work multi-select answer, synthetic data metric raw pct weighted pct ci95 low ci95 high failed reps cursor 27.6 15.2 12.9 17.8 0 github copilot 33.5 57.1 53.6 60.6 0 {'n': 4000, 'n eff': 683.0, 'design effect': 5.86, 'w min': 0.05, 'w max': 15.0, 'iterations': 32, 'converged': True} Now compare all four views in one table. The raw column is the unweighted sample. The three scenario columns are the same respondents reweighted toward three different teams. | View | Cursor share 95% interval | Copilot share 95% interval | Effective n | Design effect | Largest weight | |---|---|---|---|---|---| | Raw sample | Not listed | Not listed | 4,000 | 1.00 | 1.00 | | Balanced team | Not listed | Not listed | Not listed | Not listed | Not listed | | Startup team | Not listed | Not listed | Not listed | Not listed | Not listed | | Enterprise team trim 0.05-15 | Not listed | Not listed | Not listed | Not listed | 15.00 | Three lessons sit in that table, and each is a reason to run the scenarios at all. First, the ranking is not stable. In the startup scenario the two intervals overlap almost completely, so the honest reading is a tie. In the balanced and enterprise scenarios Copilot leads. Second, the enterprise lead looks enormous, but it rests on far fewer effective respondents than the raw sample, and its largest weight sits at the trim ceiling, so a cautious analyst would present it as a direction rather than a measurement. Third, the raw Copilot lead differs from all three team views, which shows how an unweighted headline answers a question nobody asked. Remember that the synthetic data planted these patterns by construction. Real survey data may show weaker or stronger links between company size and tool choice, and some real links may point the other way. The method transfers, the numbers do not. Step 12: Label the Result and Tie It to Today’s Products The last step is a labeling discipline. Every weighted number leaves your notebook with five items attached: the survey and edition, the fieldwork dates, the metric sentence from Step 1, the target scenario with its source, and the interval plus effective sample size. A figure missing any of the five is a rumor with decimals. The CLI already prints the basis line, so paste it into the report whole. Basis: Stack Overflow Developer Survey 2024 fieldwork May 2024, n = 65,437 . Metric: currently uses the tool at work multi-select . Weights: raked to our Nov 2026 HR export on language, org size, experience. Result: Tool A 30.1% 95% bootstrap interval 28.0-32.3 , effective n 1,881. Caveat: self-selected respondents. Weighting repairs composition, not selection. The label also has to survive product churn, and in late 2026 both products moved quickly. Cursor’s changelog, as tracked by aiticker’s Cursor release log https://aiticker.site/tools/cursor/ , lists Build Live Apps on November 5, 2026 and Voice Chat with Agents on November 6, 2026, after the Agent SDK arrived on October 20. GitHub’s own record shows the same pace. Its November 2026 changelog https://github.blog/changelog/month/11-2026/page/3/ covers OpenAI’s GPT-6.2 Astra and GPT-6.2 Sol reaching general availability in Copilot on November 17, 2026, and GitHub’s announcement of Copilot in Xcode and JetBrains Air on November 4. Social posts on November 20 and 21 also claimed that Cursor was being renamed, and nothing official confirmed that, so treat it as a rumor and watch the product name in any future survey wave. Why does this matter for a survey share? Because a question about AI coding assistants asked a year ago bundled autocomplete, chat and agents into one answer, while today the two products differ most in their agent features, cloud workflows and model menus. A respondent who said they use Copilot in 2024 may have meant tab completion. A respondent who says they use Cursor today may mostly mean agent sessions. Weighting cannot fix a category that changed underneath the question. Pricing creates a second trap, because paid and free users answer surveys alike. GitHub’s billing documentation, via the Copilot billing reference https://docs.github.com/api/article/body?pathname=%2Fen%2Fenterprise-cloud%40latest%2Fcopilot%2Fconcepts%2Fbilling-and-usage%2Findividuals%2Fbilling , sets one AI credit at $0.01 and lists Copilot Pro at $10, Pro+ at $39 and Max at $100 a month in USD, and Copilot Free includes 2,000 code completions per month. A survey respondent on Free and one on Max both count as one Copilot user in a usage percentage. For the plan-level cost picture across tools, see the Claude Code, Cursor and Copilot pricing comparison https://shattered.io/claude-code-vs-cursor-vs-github-copilot-pricing-limits-2026/ . Model churn is just as fast. GitHub deprecated several models on November 16, 2026 and added newer ones within days, a pattern the coverage of Copilot’s Carbon and Fable 5.2 additions https://shattered.io/github-copilot-agentic-coding-carbon-fable-5-2-pro-plus-39-max-100-2026/ tracks. So record the survey’s fieldwork date next to the product state at that date, and avoid describing a 2024 respondent’s tool as the product on sale today. Common Pitfalls When Weighting Cursor vs Copilot Survey Data Nine mistakes account for most wrong weighted numbers, and they cluster by pipeline stage. Check the group that matches the stage where your result first looks suspicious. Data pitfalls 1. Treating a blank as none. In many exports an empty tool cell means the respondent skipped the question, while an explicit none option means they answered and use no tool. The synthetic file writes none for the second case. Count blanks as non-users and you inflate the denominator and shrink every share. 2. Using the wrong denominator. The loader drops unanswered rows by default. Publishers often compute shares over all respondents or over only those who saw the question. Your 31% may differ from a published 28% without either number being wrong, so confirm the denominator before you call a mismatch an error. 3. Merging two different products under one alias. Copilot can mean GitHub Copilot, Microsoft’s Copilot assistant or a vendor’s own Copilot-branded feature. The alias list in the loader accepts the short form Copilot. Remove that alias when your survey lists more than one Copilot product. Weighting pitfalls 1. Trimming too hard. A band of 0.5 to 2.0 on this data left a gap of about 7 points and never converged. Over-tight trimming gives steady weights that do not match the targets, which is a quiet bias. Always check the convergence flag. 2. Zero or empty targets. A level with a zero target but a trim floor above zero keeps pulling in opposite directions. A level with a positive target and no respondents cannot be filled. Use small positive shares and merge sparse levels before raking. 3. Raking on too many variables. Each extra variable adds constraints and inflates the largest weight. If the design effect passes 3 or 4, drop the least relevant variable before you loosen the trim. Reporting pitfalls 1. Calling a weighted share a market share. It is a share of a reweighted, self-selected sample. Say that in the label. 2. Mixing survey waves. A 2024 Stack Overflow figure and a 2026 JetBrains figure differ in fieldwork date, population and question wording. Plot them on one axis only with a loud footnote, and prefer separate panels. 3. Reading overlapping intervals as a win. In the startup scenario Cursor and Copilot overlap almost entirely, so declaring a leader would overstate the evidence. State a tie, or test the difference directly with a paired bootstrap. Troubleshooting: 10 Errors and What Fixes Them Most failures in this package trace back to three places: the targets file, the trim band and the column names. The ten symptoms below map to those causes. - RAKE FAILED with a max gap above 0.01. The weights hit the trim bounds before the margins hit the targets. Loosen --trim-lo and --trim-hi in steps, coarsen the target levels, or accept that your team differs too much from the sample for a stable estimate. - RakeError: data has levels with no target. A level in the CSV has no entry in the targets file. Look for case or spelling variants such as Javascript versus JavaScript, and normalize them with df col .str.strip and a mapping before raking. - RakeError: target levels with no respondents. You set a positive share for a level the sample never contains. Merge it into a neighboring level or move its share to Other. - RakeError: shares sum to something other than 1. Rounded percentages rarely add up. Compute the last level as one minus the others, or normalize the list in code. - KeyError: tool column not in columns. The --tool-col value does not match the header. Print list df.columns , check for hidden whitespace or a different survey edition, and read the schema file for the real name. - Every tool flag is False. The delimiter in your file is not a semicolon. Open the CSV, look at a multi-select cell, and pass the right separator to load survey with the sep argument. - ModuleNotFoundError: survey rake. You ran the command outside the project root or skipped init .py . Change to the project root, confirm the file exists, and keep using python -m . - RuntimeError: no converged replicates. Every bootstrap resample failed the raking step. The cause is the same as the first item, only repeated 500 times. Fix the trim or the targets, and the bootstrap follows. - The bootstrap crawls. Each replicate rakes 4,000 rows, which is fast, but large real files multiply the cost. Develop with --reps 100 , then run 500 or more for the final numbers. - Weighted results look wild. Check the effective sample size and the largest weight first. A share built on an effective n of a few hundred and a weight at the ceiling is fragile, and the fix is a smaller target change or a more honest caveat, not more decimals. When none of these explains a problem, shrink the case. Rake on one variable, print the weighted margin, and confirm it equals the target. Add the second variable, then the third. The step where the margin stops matching points straight at the offending variable. Advanced Tips for Cleaner Weighted Estimates Once the baseline works, five refinements pay off. None is required, and each adds a little complexity, so adopt them as the data demands. - Rake on joint cells when you know them. If your HR export gives the language-by-size table, build one combined variable such as Java-large and rake on its levels. That matches the joint distribution that margin-only raking can miss. - Swap clipping for calibration bounds. Clipping inside the loop is simple but crude. Calibration methods with built-in bounds find weights that meet the margins and stay inside a band together, which avoids the stalls seen in the enterprise scenario. - Gate subgroup reports on effective n. Before printing a share for a subgroup such as Java developers at large companies, compute the effective sample size for that subgroup and suppress the line when it falls below a threshold you set in advance. - Calibrate against your own telemetry. If your team logs real assistant usage, compare the weighted survey share with the logged share. A large gap tells you the survey sample, not your weights, is the problem. A seat audit that prices cost per active developer https://shattered.io/cursor-vs-copilot-seat-audit-cost-per-active-developer-12-steps/ supplies the logged side of that comparison. - Pair survey shares with behavior. A DuckDB adoption funnel https://shattered.io/cursor-vs-copilot-adoption-funnel-duckdb-12-steps-2026/ shows how many people move from trial to steady use, and a hands-on Cursor vs Copilot benchmark https://shattered.io/benchmark-cursor-vs-github-copilot-2026/ measures task outcomes. Survey shares, funnels and benchmarks answer three different questions, and none of them replaces the others. Two housekeeping habits also help. Save the weight vector with each respondent id after every run, so you can rebuild any chart later without rerunning the raking. And pin the random seed in the report header. A reviewer who cannot reproduce an interval to the first decimal has no reason to trust it. Finally, watch for survey weights published by the survey itself. Some editions ship a weight column, and some do not. When a weight exists, read how the publisher built it before you stack your own raking on top, because double weighting can distort results in ways that are hard to spot. When none exists, your raking is the first correction the data receives. The Complete Working Project and Run Order The package is complete once the eleven files from Step 2 exist and the five tests pass. The run order below reproduces every printed output from a clean folder. It regenerates the synthetic file with its fixed seed, runs the tests, and then runs the scenarios. source .venv/bin/activate python -m survey rake.synth data/synthetic survey.csv python -m pytest -q survey rake/tests python -m survey rake.cli --csv data/synthetic survey.csv --tool-col ai tools work \ --targets data/targets.json --scenario all python -m survey rake.cli --csv data/synthetic survey.csv --tool-col ai tools work \ --targets data/targets.json --scenario enterprise --trim-lo 0.05 --trim-hi 15 The CLI exits with status 2 when any scenario fails to converge, so you can drop the command into a scheduled job or a CI step and let a failed rake stop the pipeline. To move to real data, replace the CSV path, set --tool-col to the column named in the survey’s schema file, recode the three trait columns to match your target levels, and fill in the ledger. The code does not change. The structure also extends cleanly. A fourth module could write weights back to disk, a fifth could draw the sensitivity table as a chart, and a notebook could call build report for any tool flag you add to the alias list. If you want more tooling guides in the same vein, the coding and developer tools section https://shattered.io/category/software/ collects them. Keep the synthetic results out of anything you publish as a finding. They exist to prove the pipeline works and to show how sensitive a ranking can be, not to describe the market. The first number you publish should come from a real file, a cited target table and a label with all five required items. FAQ Does this method give the market share of Cursor vs Copilot? No. It gives the share of a reweighted survey sample that reports using each tool. Market share needs data on all users or all revenue, and a survey has neither. The label from Step 12 keeps that distinction visible. Which variables should I rake on? Pick variables that plausibly affect tool choice and that you can measure on your own team. Language, company size and seniority are the usual three. Add more only if the design effect stays reasonable and every level still has respondents. How many respondents do I need? No fixed number works for every survey. Watch the effective sample size after weighting and keep every target level populated by at least a few dozen respondents. When the design effect climbs past 3 or 4, the estimate is carrying too much correction. Can I run this on the real Stack Overflow or JetBrains file? Yes, with care. Download the data, read the license and the schema, point --tool-col at the right column, recode your trait columns, and fill in the ledger. Stack Overflow’s pages state ODbL 1.0 for the data, and JetBrains lists its surveys by year on its research hub. Why bootstrap instead of a formula? Weighted shares carry extra variance that plain proportion formulas ignore, and raking is iterative, so no short closed form exists. Resampling and re-raking handles both without extra algebra. Why does the Cursor share fall while the Copilot share rises in two scenarios? Both effects come from the planted structure. In the synthetic data, Cursor leans toward small and solo shops and Copilot leans toward large organizations. When the target team has more large-company respondents, the weights shift toward people who favor Copilot. How often should I rerun it? Rerun it when a new survey wave is published or when your team’s mix changes. JetBrains lists an edition for every year since 2017, and Stack Overflow publishes annually, so a yearly refresh matches the source cadence. Record each run’s fieldwork date so old and new waves never blend.