Mining Architectural Quality Under Agentic AI Adoption

A staggered difference-in-differences study of 151 open-source Java repositories, 74 with detectable agentic AI adoption and 77 propensity-matched controls, found that agentic AI coding tools left total architectural smell counts essentially unchanged (+1.1%, p = 0.82) while lines of code grew 12.8% (p = 0.003), producing a 6.7% decline in architectural smell density (p = 0.004) that the authors attribute to a denominator effect rather than architectural improvement. The study, submitted to arXiv on 11 June 2026, drew on 1,811 monthly Arcan snapshots across a 13-month per-repository window and used the Borusyak imputation estimator, with wild cluster bootstrap, Lee bounds and stale-observation sensitivity checks corroborating the pattern and flat pre-trends (Wald p = 0.90). The authors conclude that density-normalized outcomes can mislead when treatment affects system size, and that raw counts plus explicit decomposition are required for causal mining studies of AI tool adoption; the replication package and curated 151-repository monthly panel are publicly available.

Computer Science Software Engineering Submitted on 11 Jun 2026 Title:Mining Architectural Quality Under Agentic AI Adoption: A Causal Study of Java Repositories View PDF https://arxiv.org/pdf/2606.13298 HTML experimental https://arxiv.org/html/2606.13298v1 Abstract:AI coding tools are now used by a majority of developers, and agentic use of these tools has popularized the practice colloquially called "vibe coding". Yet causal evidence on their effect on software architecture is scarce. Prior causal work has measured code-level outcomes complexity, static analysis warnings ; whether such degradation propagates to architecture-level outcomes remains unknown. We mine 151 open-source Java repositories, 74 with detectable agentic AI adoption identified via configuration files and Co-Authored-By commit trailers and 77 propensity-matched controls, across a 13-month per-repository window yielding 1,811 monthly Arcan snapshots. We estimate the causal effect of adoption on architectural smell density ASD with a staggered difference-in-differences design and the Borusyak imputation estimator, applying a causal design recently used for code-level metrics to the architecture level. Total smell counts are essentially unchanged +1.1%, p = 0.82 while lines of code grow +12.8% p = 0.003 ; the resulting 6.7% ASD decline p = 0.004 is therefore a denominator effect rather than an architectural improvement. Per-type estimates and robustness checks wild cluster bootstrap, Lee bounds, stale-observation sensitivity corroborate the pattern; pre-trends are flat Wald p = 0.90 , consistent with parallel trends. Density-normalized outcomes can mislead when treatment affects system size: raw counts and explicit decomposition are required for causal mining studies of AI tool adoption. The complete replication package, including the curated 151-repository monthly panel, is publicly available. Additional Features References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .