# Measuring AI productivity yourself with gh, jq, and Git > Source: > Published: 2026-06-19 15:26:30+00:00 # Measuring AI productivity: Why your numbers are flat and your rework is up Across engineering organizations AI usage is up ~65% but PR throughput rose only 7.76%. Here's why your productivity dashboard can't tell motion from delivery, and four metrics you can baseline next week with nothing but gh, jq, and git. If you've had the misfortune of opening social media recently, you've probably seen it everywhere. Tokenomics has come and the Reapers are here. Tokenomics refers to the concept that while many of us have started to use AI tools and are feeling productive by generating mountains of code, shipping refactors, and going to bed while agents work overnight, the mergeable output and shipped software to production has not kept pace. What it's describing is token yield: what is the outcome of all of this generation for the business, for your customers, and for your own sanity? Even the Linux Foundation has announced its [Tokenomics Foundation](https://www.tokeneconomics.com/) as well as its upcoming conference [Tokenomicon](https://www.linuxfoundation.org/press/linux-foundation-announces-tokenomicon-a-new-conference-for-the-economics-of-ai), a brilliant hat tip to the [Necronomicon](https://en.wikipedia.org/wiki/Necronomicon). They named the conference for AI budgets after the Book of the Dead, which feels about right. These jokes exist because the bill is real and it hurts. AI is now one of the fastest-growing line items on an engineering budget and in the Linux Foundation's own words, "the discipline to measure and govern that spend has not kept pace." ## Someone finally measured[#](#someone-finally-measured) [DX ran a 400 organization study](https://getdx.com/blog/ai-productivity-gains-are-10-percent-not-10x/) from late 2024 through early 2026 and found that pull request throughput was only up 7.76% while AI usage rose ~65%. [METR surveyed 349 technical workers in May](https://metr.org/blog/2026-05-11-ai-usage-survey/), 87 software engineers among them, and the median self-report was two times more valuable work and three times faster. However, METR's own staff gave the lowest estimates of any subgroup in their survey. They believe it's because their team has read about the perception gap research and had an observation bias. When the people who are most familiar with the measurement data don't trust their own feelings, that tells you something. In [another 2025 METR study](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/), 16 experienced open source developers worked on 246 real tasks on their own repos. Each task was randomly assigned to either allow or forbid AI use. Before starting, they predicted that AI would make them 24% faster. After finishing, they believed it had made them 20% faster. They took 19% longer. The perception error survived contact with the real experience. Surveys can't save you. METR followed up that study [with another in February 2026](https://metr.org/blog/2026-02-24-uplift-update/) with 57 developers and over 800 tasks. They discovered that the slowdown shrank or in some cases even reversed, though the confidence intervals cross zero and METR itself claims only "some evidence for speedup." Three key things had changed in this time period. The models got better, Opus 4.5 had shipped mid-study. Developers got more practice with these tools. And the cohort itself changed, with 47 newly recruited developers working on a more diverse set of repos. This isn't a knock on the research, it's the finding. Every number in this field is a snapshot. Specific set of people with a specific set of repos using specific models to accomplish their outcomes. Borrowed conclusions don't transfer into your company. So the only productivity number worth anything is the one you measure yourself. ## Why does my dashboard say we're productive?[#](#why-does-my-dashboard-say-were-productive) We all know that unconstrained AI produces more code and bigger diffs, so the cost moves towards review. [Faros measured](https://www.faros.ai/ai-productivity-paradox) that review time was up 91% in teams with high AI adoption. That means your throughput chart counting when the PR opens isn't helpful. Like a tokenmaxxing leaderboard, while it may motivate people to start using AI it isn't tracking to useful outcomes. What's concerning is when this review queue backs up far enough. Teams do the obvious thing. They stop reviewing. A separate Faros report found that [31.3% more PRs are merged with no review at all](https://pages.faros.ai/hubfs/AI_Engineering_Report_2026_The_Acceleration_Whiplash_Faros.pdf). [CircleCI found another interesting fact](https://circleci.com/landing-pages/assets/2026-state-of-software-delivery-report.pdf). The median team pushed 15% more throughput to feature branches while their main-branch throughput dropped 7%. Even the top 10% of teams, with feature branch throughput up ~50%, saw main-branch activity stay roughly flat. And main branch merge success now sits at 70.8%, the lowest in over 5 years. Motion is up, yet delivery is flat. And all the tokens in between are on fire. "It's the J-curve," they'll shout. "You're measuring in the dip!" This will remain unfalsifiable if we never define what climbing out of the J-curve looks like. A real J-curve claim requires a dated exit criterion. And if no one will write down when the curve turns and what number proves it, it's not a useful model. It's an excuse with a graph that's leading you to burn tokens. ## What to do about it[#](#what-to-do-about-it) You can't prove a delta without a before. And most teams deleted their "before" the day they rolled out these new tools. It's critical that you first answer the question, "What was our baseline?" Same-engineer baselining where you compare each engineer to their own pre-AI history, not to other teams, is a useful technique. This way it can kill confounds like tenure, team changes, seasonality, or even self-selection (early adopters were likely your strongest engineers). The proof that this works is from [DX's financial services case](https://getdx.com/blog/ai-measurement-hub/) where AI users had an increase of 30% PR throughput year over year against their own baseline and non-AI users had a 5% increase at the same company. To get a baseline next week, here's what you do. For each engineer, track four items: Cycle time, review time, rework within 30 days of merge, and defect escape rate. Then segment AI-assisted work against the rest. **Make sure** you aggregate at the team level. Never use this for individual performance. The moment an engineer thinks that this is to track their personal productivity, it looks like a report card and your data will die. None of this needs a vendor. If your code lives on GitHub or GitLab, the data already exists. The whole stack in the examples below uses GitHub CLI (`gh` ), `jq` , `git` , and a cron job. ### Cycle time[#](#cycle-time) Cycle time is the time from opening a pull request to merging it. Use the median, not the mean, otherwise one stale PR will ruin your analysis. ``` gh pr list --state merged --limit 100 \ --json createdAt,mergedAt \ | jq '[.[] | (.mergedAt|fromdateiso8601) - (.createdAt|fromdateiso8601)] | sort | {merged_prs: length, median_cycle_hours: ((.[length/2|floor]) / 3600 | floor)}' ``` Two things will mess with this number: Draft PRs and Bots #### Draft PRs[#](#draft-prs) You only need to do this if your team leverages draft PRs. The clock above is designed to start when the PR is created. If your team opens drafts early to run CI, have AI agents review in remote, or share progress, the timer will run while that work is still half done and your cycle time will look worse than it is. GitHub records the moment a draft flips to ready for review so you can measure from that instead: ``` gh api graphql -f owner="$(gh repo view --json owner -q .owner.login)" \ -f repo="$(gh repo view --json name -q .name)" -f query=' query($owner: String!, $repo: String!) { repository(owner: $owner, name: $repo) { pullRequests(states: MERGED, last: 100) { nodes { createdAt mergedAt timelineItems(itemTypes: READY_FOR_REVIEW_EVENT, first: 1) { nodes { ... on ReadyForReviewEvent { createdAt } } } } } } }' \ | jq '[.data.repository.pullRequests.nodes[] | ((.timelineItems.nodes[0].createdAt // .createdAt) | fromdateiso8601) as $start | (.mergedAt | fromdateiso8601) - $start] | sort | {merged_prs: length, median_cycle_hours_from_ready: ((.[length/2|floor]) / 3600 | floor)}' ``` #### Bots[#](#bots) Your team is likely running at least Dependabot. And its PRs either merge themselves in minutes or sometimes sit ignored for months. Either way, you don't want to measure a robot here, exclude them with: `--search "-author:app/dependabot"` . ### Review time[#](#review-time) One query that gets you two numbers. How long code waits for human eyes and how much of it merges with no eyes at all. Remember that 31.3% from Faros? This is how you measure your own: ``` gh pr list --state merged --limit 100 \ --json createdAt,reviews \ | jq '{ total: length, merged_with_zero_reviews: [.[] | select((.reviews|length)==0)] | length, median_hours_to_first_review: ([.[] | select((.reviews|length) > 0) | ((.reviews | map(.submittedAt) | sort | first | fromdateiso8601) - (.createdAt|fromdateiso8601))] | sort | (.[length/2|floor]) / 3600 | floor) }' ``` One caveat: first review doesn’t always mean a thorough review. A rubber-stamp approval counts the same as someone who read every single line. In this analysis, that's fine because you're trying to watch for a trend, not grading the actual reviews. But you should watch the two numbers together. If reviews are arriving faster while more PRs merge with no review at all, you may not be speeding up—you’ve just stopped reviewing. ### Rework[#](#rework) Rework can be measured in many different ways, here is an approach that's fast and requires nothing but git history, python, and a splash of cron. The goal is to answer the question “of all the code that we deleted this month, how much was brand new?” Old code getting deleted is normal and celebrated. Three week old code getting deleted means you paid to write it, paid to review it, and then paid a third time to replace it. How the measurement works: Walk every commit from the last 30 days. For each line a commit deleted, ask git blame how old that line was. If it was less than 30 days old when it was deleted, count it as rework. Young deletions divided by all deletions, that's the rate. This is the metric that dashboards charge for, and under the hood, it's some version of these same three git commands. Save this script as `rework_rate.py` at the root of your repo and run it with Python 3. One word of warning: it’s slow on big and busy repos because it runs git blame once for every block of deleted lines, so put it in a monthly cron. **Note**: 30 days is the default. If your team is high velocity, run a 14 day version. ``` bash #!/usr/bin/env python3 """Rework rate: share of deleted lines under AGE days old when deleted. Usage: python3 rework_rate.py [window_days] [age_days] (run at repo root)""" import re, subprocess, sys WINDOW = sys.argv[1] if len(sys.argv) > 1 else "30" AGE = int(sys.argv[2]) if len(sys.argv) > 2 else 30 def git(*args): return subprocess.run(["git", *args], capture_output=True, text=True).stdout young = total = 0 for entry in git("log", f"--since={WINDOW} days ago", "--no-merges", "--pretty=%H %ct").splitlines(): sha, ts = entry.split() deleted_at = int(ts) for f in git("diff-tree", "--no-commit-id", "--name-only", "-r", sha).splitlines(): diff = git("diff", "-U0", f"{sha}^", sha, "--", f) for m in re.finditer(r"^@@ -(\d+)(?:,(\d+))?", diff, re.M): start, count = int(m.group(1)), int(m.group(2) or 1) if count == 0: continue blame = git("blame", "--line-porcelain", f"-L{start},+{count}", f"{sha}^", "--", f) for a in re.finditer(r"^author-time (\d+)", blame, re.M): total += 1 if deleted_at - int(a.group(1)) < AGE * 86400: young += 1 if total: print(f"deleted lines: {total} | young (<{AGE}d): {young} | " f"rework rate: {young/total:.1%}") else: print("no deletions in window") ``` I ran this while writing this post on our monorepo. 836 commits and 215,653 deleted lines later: 27%. One in four lines we deleted last month was under 30 days old. At a 14-day threshold it’s 20.8%, because most rework happens fast once AI-Pilled. Don't go looking for an industry benchmark because there isn't one and you don't need one. All repos churn differently, the script doesn't follow file renames and one big formatting pass will read as a spike. None of that matters because the number only means something against your own history. Run it monthly, chart it next to your AI adoption, watch the direction, not the absolute value. Then pair it with cycle time because that pairing is the payoff of this entire section. Throughput up and rework flat means the gains are real. Throughput up and rework climbing means you're reshipping the same work twice and trying to call it velocity. Two numbers at one glance and you'll have answered the question that we opened this post with. ### Defect escape rate[#](#defect-escape-rate) This final metric is different. The first three run today against history that you already have. This one asks your team to pick up a habit first and it pays you every month after. It's worth it. Defect escape rate is the share of your bugs that customers found before you did. Every bug gets caught somewhere. The ones your tests and reviews catch are cheap. The ones that escape to production are expensive. If this rate climbs alongside your throughput, you're shipping faster than your safety net can check. You cannot reconstruct the baseline later. Once AI is embedded in your workflows, it is very hard to get the pre-AI signal, so start soon. To track this in your issue tracker: label every `bug` . Bugs that made it to production also get `escaped` as a label. Keep that up for a month and the rate can be computed with a few commands depending on your tracker. #### GitHub Issues[#](#github-issues) Bash is all you need: ``` MONTH="2026-05" ALL=$(gh issue list --state all --label bug \ --search "created:${MONTH}-01..${MONTH}-31" --limit 500 --json number | jq length) ESCAPED=$(gh issue list --state all --label bug --label escaped \ --search "created:${MONTH}-01..${MONTH}-31" --limit 500 --json number | jq length) echo "$MONTH defect escape rate: $ESCAPED/$ALL" ``` #### Jira[#](#jira) In Jira, the same two counts are two lines of JQL (Jira Query Language), paste it into the search bar. The count will then sit at the bottom of the results. All bugs created in the month: ``` issuetype = Bug AND created >= "2026-05-01" AND created < "2026-06-01" ``` And of those, the ones that escaped to production: ``` issuetype = Bug AND labels = escaped AND created >= "2026-05-01" AND created < "2026-06-01" ``` If your Jira setup already has a "found in environment" field on bugs you don't need a new label. Use the same queries, swap the label clause for the field. Counts now come from a new dedicated endpoint: ``` curl -s -X POST 'https://YOUR-SITE.atlassian.net/rest/api/3/search/approximate-count' \ -u 'you@company.com:YOUR_API_TOKEN' \ -H 'Content-Type: application/json' \ -d '{"jql": "issuetype = Bug AND labels = escaped AND created >= 2026-05-01 AND created < 2026-06-01"}' ``` Returns {"count": N} — run once per query, divide #### Linear[#](#linear) Two saved views filtered by the `bug` label and created date, one with the `escaped` label added. Each view shows its count. For the cron, one GraphQL call returns both numbers: ``` curl -s https://api.linear.app/graphql \ -H "Authorization: $LINEAR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"query": "{ all: issues(first: 250, filter: { labels: { name: { eq: \"bug\" } }, createdAt: { gte: \"2026-05-01\", lt: \"2026-06-01\" } }) { nodes { id } } escaped: issues(first: 250, filter: { and: [ { labels: { name: { eq: \"bug\" } } }, { labels: { name: { eq: \"escaped\" } } } ], createdAt: { gte: \"2026-05-01\", lt: \"2026-06-01\" } }) { nodes { id } } }"}' \ | jq '{all: (.data.all.nodes|length), escaped: (.data.escaped.nodes|length)}' ``` Pages cap at 250—if you file more than 250 bugs a month, you have my sympathy and a pagination problem. #### We won't label[#](#we-wont-label) I get it and git can still help give a little signal: ``` echo "reverts last 30d: $(git log --since='30 days ago' --oneline --grep='^Revert' | wc -l)" echo "merges last 30d: $(git log --since='30 days ago' --oneline --merges | wc -l)" ``` Returns two numbers, something like `reverts: 3, merges: 120` . Divide them and you have your panic rate. Treat this as a floor and not the full picture. Most production bugs get fixed forward rather than reverted, so reverts will only catch the loudest escapes, the ones bad enough someone hit undo. That's why the number is useful. If reverts are climbing as a share of merges month over month, bugs are reaching production faster than your reviews can catch them. At least you've found that out without having to ask anybody to do data labeling. ### Segmenting AI-assisted work[#](#segmenting-ai-assisted-work) Depending on your team's tool chain, there are different approaches to doing the segmentation of AI-assisted work. - Add an `ai-assisted` label through a checkbox on your PR template - Skip labeling and count the `Co-Authored-By` lines that tools like Claude Code and Copilot add to your commit messages ``` # label-based gh pr list --state merged --limit 100 --json labels \ | jq '{ai_assisted: [.[] | select(.labels | map(.name) | index("ai-assisted"))] | length, total: length}' # trailer-based (Claude Code, Copilot, etc. already write these) git log --since="30 days ago" --grep="Co-Authored-By: Claude" --oneline | wc -l ``` Put this into a weekly cron, append each run to a CSV, and chart it. Everything here works the same on GitLab with `glab` replaced for `gh` . Finally, when a vendor pitches you next quarter, you'll be able to judge it against your own team's baseline instead of relying on their demo data. ## Something upstream is wrong[#](#something-upstream-is-wrong) With flat throughput, rising rework, and feature branches full of motion that never reach main, we're not having a measurement failure. The instrumentation is working. It's reporting a real condition: something upstream of these metrics is wrong. The gains disappear between the feature branch and main because the agent never actually understood your system. Every win got eaten by steering, correcting, doom loops, and redoing. When an agent doesn't understand how your system rolls out, it pushes code that breaks in production, and the ICs spend the next sprint dealing with the thrash instead of working the review backlog or shipping the next feature. It doesn't matter whether that code came from a quick prompt or an overnight long-running agent. Thrash is thrash. So get your baseline this week and watch what the numbers do. And when throughput comes back flat while everyone on the team swears they're faster, you'll be left with the question your dashboard can't answer: what does your agent actually know about your system?