Why AI-Generated GIS Code Can Run Successfully and Still Be Wrong

Geospatial code can execute without errors and produce clean-looking results while being methodologically wrong, a problem amplified by AI coding agents generating GIS workflows from natural-language instructions. Developer Daniel J. Wright built GeoAI Skills, a set of 18 reusable Agent Skills that encode safeguards such as CRS checks, projection validation, and spatially blocked cross-validation to prevent silent failures in AI-generated geospatial analysis.

Geospatial code has an unusual failure mode: it can execute without errors, produce a clean-looking result, and still be methodologically wrong . This problem is becoming more important as AI coding agents generate complete GIS workflows from short natural-language instructions. The issue is not always that an AI agent does not know GeoPandas, rasterio, PostGIS, Earth Engine, or ArcPy. In many cases, it knows the correct API and can produce executable code. The problem is that API correctness is not the same as geospatial correctness. Consider a simple buffer operation: buffered = gdf.buffer 1000 The code looks reasonable. It runs successfully and produces new geometries. But what does 1000 mean? When the dataset uses a geographic coordinate reference system such as EPSG:4326 , its coordinate units are degrees rather than metres. The operation may still return valid geometries, but they do not represent a 1,000-metre buffer . A safer workflow makes the spatial assumptions explicit: if gdf.crs is None: raise ValueError "The input dataset has no CRS." if gdf.crs.is geographic: projected crs = gdf.estimate utm crs if projected crs is None: raise ValueError "A suitable projected CRS could not be determined." gdf = gdf.to crs projected crs buffered = gdf.buffer 1000 Even this pattern is not universally correct. A locally estimated UTM projection may be unsuitable for datasets that span multiple zones, countries, or continents. The important point is not one specific code snippet. A defensible workflow must ask: Is the CRS defined? Are the coordinate units appropriate? Is the projection suitable for the geographic extent? How will the output be verified? Without those checks, plausible output can hide an invalid spatial operation. Spatial machine learning has a similar problem. A common model evaluation starts with a random train/test split: python from sklearn.model selection import train test split X train, X test, y train, y test = train test split X, y, test size=0.2, random state=42, For independent observations, this can be a reasonable starting point. Spatial observations, however, are often correlated with nearby observations. A random split can place neighbouring samples in both the training and test sets. The model then appears to generalize, while it may only be exploiting local spatial similarity. This can produce an impressive accuracy score that collapses when the model is applied to a genuinely new region. A more defensible evaluation may require: Spatially blocked validation often produces a lower score. That lower score may be the more honest result. Rendering is not verification Maps introduce another form of silent failure. A map can render correctly while using: A change-detection map can look convincing even when the source rasters are misregistered, collected in different seasons, or processed at incompatible levels. Successful rendering proves that the software produced an image. It does not prove that the analysis is valid. What an AI agent needs beyond API knowledge A general-purpose coding agent may know which function to call. It does not necessarily know when a workflow should stop. For geospatial tasks, an agent needs explicit operational rules: These are not optional improvements added after the analysis. They are part of the analytical method itself. I built GeoAI Skills to encode these safeguards as reusable Agent Skills. The current public preview contains 18 skills covering: The objective is to make assumptions visible, require verification, and fail loudly when a workflow is not defensible. A land-suitability request, for example, may involve data preparation, remote sensing, terrain analysis, multi-criteria decision analysis, and cartographic delivery. GeoAI Skills uses a central orchestrator to route each stage to the relevant specialist skill while maintaining shared safeguards such as CRS checks, leakage prevention, uncertainty reporting, and verification. The suite also includes an arcgis-pro-automation skill. It works with my open-source arcgis-mcp-bridge project to support controlled local ArcPy execution. The integration covers workflows involving: Because ArcPy operations can modify real projects and datasets, the execution model includes path restrictions and confirmation gates for mutating or destructive actions. The goal is not unrestricted automation. It is controlled automation with explicit boundaries. A skill is not useful if the agent does not activate it for the right request—or activates it for unrelated requests. The repository currently contains 131 typed evaluation scenarios across the 18 skills. These scenarios include: For a frozen 17-skill, 120-case routing suite using Claude Code 2.1.214 and Claude Sonnet 5, the published results were: 100% routing precision 92.86% routing recall 92.5% full-route accuracy These results have an important limitation. They measure whether the expected skills were selected for that exact runtime, model, and evaluation suite. They do not prove that every generated analysis is correct, and they are not universal compatibility or answer-quality claims. The eighteenth skill, arcgis-pro-automation, was added after the frozen benchmark and is not included in those headline figures. Routing evidence, behavioural correctness, and real-world analytical quality should be evaluated separately. Trying the public preview The complete suite can be explored and installed interactively with: npx skills add muend/geoai-skills A single specialist skill can also be installed independently: npx skills add muend/geoai-skills --skill postgis-spatial-sql GitHub repository: https://github.com/muend/geoai-skills https://github.com/muend/geoai-skills Skills collection: https://www.skills.sh/muend/geoai-skills https://www.skills.sh/muend/geoai-skills The project is still an early public preview. I am particularly interested in reproducible examples of: What is the most common silent geospatial failure you have encountered?