{"slug": "why-ai-generated-gis-code-can-run-successfully-and-still-be-wrong", "title": "Why AI-Generated GIS Code Can Run Successfully and Still Be Wrong", "summary": "Geospatial code can execute without errors and produce clean-looking results while being methodologically wrong, a problem amplified by AI coding agents generating GIS workflows from natural-language instructions. Developer Daniel J. Wright built GeoAI Skills, a set of 18 reusable Agent Skills that encode safeguards such as CRS checks, projection validation, and spatially blocked cross-validation to prevent silent failures in AI-generated geospatial analysis.", "body_md": "Geospatial code has an unusual failure mode: it can execute without errors, produce a clean-looking result, and still be **methodologically wrong**.\n\nThis problem is becoming more important as AI coding agents generate complete GIS workflows from short natural-language instructions.\n\nThe issue is not always that an AI agent does not know GeoPandas, rasterio, PostGIS, Earth Engine, or ArcPy. In many cases, it knows the correct API and can produce executable code.\n\nThe problem is that API correctness is not the same as geospatial correctness.\n\nConsider a simple buffer operation:\n\n```\n`buffered = gdf.buffer(1000)`\n```\n\nThe code looks reasonable. It runs successfully and produces new geometries.\n\nBut what does 1000 mean?\n\nWhen the dataset uses a geographic coordinate reference system such as **EPSG:4326**, its coordinate units are degrees rather than metres. The operation may still return valid geometries, but they do not represent a **1,000-metre buffer**.\n\nA safer workflow makes the spatial assumptions explicit:\n\n```\nif gdf.crs is None:\n    raise ValueError(\"The input dataset has no CRS.\")\n\nif gdf.crs.is_geographic:\n    projected_crs = gdf.estimate_utm_crs()\n\n    if projected_crs is None:\n        raise ValueError(\"A suitable projected CRS could not be determined.\")\n\n    gdf = gdf.to_crs(projected_crs)\n\nbuffered = gdf.buffer(1000)\n```\n\nEven this pattern is not universally correct. A locally estimated UTM projection may be unsuitable for datasets that span multiple zones, countries, or continents.\n\nThe important point is not one specific code snippet. A defensible workflow must ask:\n\n**Is the CRS defined?**\n\nAre the coordinate units appropriate?\n\nIs the projection suitable for the geographic extent?\n\nHow will the output be verified?\n\nWithout those checks, plausible output can hide an invalid spatial operation.\n\nSpatial machine learning has a similar problem.\n\nA common model evaluation starts with a random train/test split:\n\n``` python\nfrom sklearn.model_selection import train_test_split\n\nX_train, X_test, y_train, y_test = train_test_split(\n    X,\n    y,\n    test_size=0.2,\n    random_state=42,\n)\n```\n\nFor independent observations, this can be a reasonable starting point.\n\nSpatial observations, however, are often correlated with nearby observations. A random split can place neighbouring samples in both the training and test sets.\n\nThe model then appears to generalize, while it may only be exploiting local spatial similarity.\n\nThis can produce an impressive accuracy score that collapses when the model is applied to a genuinely new region.\n\n**A more defensible evaluation may require:**\n\nSpatially blocked validation often produces a lower score.\n\nThat lower score may be the more honest result.\n\nRendering is not verification\n\nMaps introduce another form of silent failure.\n\n**A map can render correctly while using:**\n\nA change-detection map can look convincing even when the source rasters are misregistered, collected in different seasons, or processed at incompatible levels.\n\nSuccessful rendering proves that the software produced an image.\n\nIt does not prove that the analysis is valid.\n\nWhat an AI agent needs beyond API knowledge\n\nA general-purpose coding agent may know which function to call. It does not necessarily know when a workflow should stop.\n\n**For geospatial tasks, an agent needs explicit operational rules:**\n\nThese are not optional improvements added after the analysis.\n\nThey are part of the analytical method itself.\n\n**\n\n**\n\nI built GeoAI Skills to encode these safeguards as reusable Agent Skills.\n\n**The current public preview contains 18 skills covering:**\n\nThe objective is to make assumptions visible, require verification, and fail loudly when a workflow is not defensible.\n\nA land-suitability request, for example, may involve data preparation, remote sensing, terrain analysis, multi-criteria decision analysis, and cartographic delivery.\n\nGeoAI Skills uses a central orchestrator to route each stage to the relevant specialist skill while maintaining shared safeguards such as CRS checks, leakage prevention, uncertainty reporting, and verification.\n\n**\n\n**\n\nThe suite also includes an arcgis-pro-automation skill.\n\nIt works with my open-source **arcgis-mcp-bridge** project to support controlled local ArcPy execution.\n\n**The integration covers workflows involving:\n**\n\nBecause ArcPy operations can modify real projects and datasets, the execution model includes path restrictions and confirmation gates for mutating or destructive actions.\n\nThe goal is not unrestricted automation.\n\nIt is controlled automation with explicit boundaries.\n\n**\n\n**\n\nA skill is not useful if the agent does not activate it for the right request—or activates it for unrelated requests.\n\nThe repository currently contains 131 typed evaluation scenarios across the 18 skills.\n\n**These scenarios include:**\n\nFor a frozen 17-skill, 120-case routing suite using Claude Code 2.1.214 and Claude Sonnet 5, the published results were:\n\n**100% routing precision\n92.86% routing recall\n92.5% full-route accuracy**\n\nThese results have an important limitation.\n\nThey measure whether the expected skills were selected for that exact runtime, model, and evaluation suite. They do not prove that every generated analysis is correct, and they are not universal compatibility or answer-quality claims.\n\nThe eighteenth skill, arcgis-pro-automation, was added after the frozen benchmark and is not included in those headline figures.\n\nRouting evidence, behavioural correctness, and real-world analytical quality should be evaluated separately.\n\nTrying the public preview\n\nThe complete suite can be explored and installed interactively with:\n\n`npx skills add muend/geoai-skills`\n\nA single specialist skill can also be installed independently:\n\n`npx skills add muend/geoai-skills --skill postgis-spatial-sql`\n\n**GitHub repository:**\n\n[https://github.com/muend/geoai-skills](https://github.com/muend/geoai-skills)\n\n**Skills collection:**\n\n[https://www.skills.sh/muend/geoai-skills](https://www.skills.sh/muend/geoai-skills)\n\n**The project is still an early public preview.**\n\n**I am particularly interested in reproducible examples of:**\n\n**What is the most common silent geospatial failure you have encountered?**", "url": "https://wpnews.pro/news/why-ai-generated-gis-code-can-run-successfully-and-still-be-wrong", "canonical_source": "https://dev.to/muend/why-ai-generated-gis-code-can-run-successfully-and-still-be-wrong-473k", "published_at": "2026-07-23 15:01:21+00:00", "updated_at": "2026-07-23 15:33:55.267766+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools", "computer-vision", "machine-learning"], "entities": ["Daniel J. Wright", "GeoAI Skills", "GeoPandas", "PostGIS", "Earth Engine", "ArcPy", "sklearn"], "alternates": {"html": "https://wpnews.pro/news/why-ai-generated-gis-code-can-run-successfully-and-still-be-wrong", "markdown": "https://wpnews.pro/news/why-ai-generated-gis-code-can-run-successfully-and-still-be-wrong.md", "text": "https://wpnews.pro/news/why-ai-generated-gis-code-can-run-successfully-and-still-be-wrong.txt", "jsonld": "https://wpnews.pro/news/why-ai-generated-gis-code-can-run-successfully-and-still-be-wrong.jsonld"}}