{"slug": "agentic-coding-without-the-cloud-evaluating-open-weight-large-language-models-on", "title": "Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks", "summary": "A new open-source framework from University College London (UCL) evaluates open-weight large language models on longitudinal data preparation tasks without sending data to the cloud. Benchmarking models up to 31-35B parameters achieved up to 87.9% average task completion on 20 tasks creating 102 variables, showing promise for AI-assisted data preparation in governance-restricted research settings.", "body_md": "arXiv:2607.21482v2 Announce Type: replace-cross\nAbstract: Large language models (LLMs) and agents are now widely used tools in code development, with data typically sent to third-party cloud-based models. Their adoption in research using personal data is constrained by governance requirements that typically prohibit data transmission to external services. Locally deployable open-weight models offer an alternative since sensitive data never leave the local environment. We introduce an open-source framework for evaluating the efficacy of AI agents powered by open-weight LLMs on one of the most persistent bottlenecks in research on longitudinal population studies: data preparation. The framework comprises: a curated ground-truth dataset (cleaning scripts preparing six sweeps of data from a British cohort study), task definitions encompassing tasks such as category harmonization and multi-wave merging, and automated routines for evaluating the LLM-produced R code and outputted data. We benchmark LLMs across the (consumer grade) deployment spectrum to assess their efficacy in 20 data preparation tasks (creation of 102 variables). Current state-of-the-art, 31-35B parameter models almost saturated our benchmark ('average task completion' up to 87.9%). The performance of open-weight LLMs running on consumer-grade hardware shows promise of a viable path toward AI-assisted data preparation in governance-restricted research settings. Our framework is publicly available at: https://github.com/UCL-ARC/RRBench.", "url": "https://wpnews.pro/news/agentic-coding-without-the-cloud-evaluating-open-weight-large-language-models-on", "canonical_source": "https://www.machinebrief.com/news/agentic-coding-without-the-cloud-evaluating-open-weight-larg-8fio", "published_at": "2026-07-27 04:00:00+00:00", "updated_at": "2026-07-27 05:28:10.079269+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-research", "ai-tools"], "entities": ["University College London", "UCL-ARC", "RRBench"], "alternates": {"html": "https://wpnews.pro/news/agentic-coding-without-the-cloud-evaluating-open-weight-large-language-models-on", "markdown": "https://wpnews.pro/news/agentic-coding-without-the-cloud-evaluating-open-weight-large-language-models-on.md", "text": "https://wpnews.pro/news/agentic-coding-without-the-cloud-evaluating-open-weight-large-language-models-on.txt", "jsonld": "https://wpnews.pro/news/agentic-coding-without-the-cloud-evaluating-open-weight-large-language-models-on.jsonld"}}