Every engineer has uttered the phrase, "But it works on my machine."
I hit a wall that perfectly encapsulates why that phrase is a trap. The application ran flawlessly in my local development environment but immediately crashed upon deployment to the cloud.
Here is a post-mortem of the two major blockers I faced during deployment, how I diagnosed them, and the pragmatic resolutions that got the agent into production.
Challenge 1: The KeyError: '_type' Mystery
The Symptom
The application deployed successfully, but the moment it tried to initialize the ChromaDB vector database, it crashed with a glaring KeyError: '_type'.
The Investigation (Root Cause Analysis)
My first instinct was to check the code, but the initialization logic was identical in both environments. The issue had to be environmental. I started comparing my local setup against the Docker container:
Local Machine: Windows, Python 3.13.
Docker Container: Linux, Python 3.11.
I dug into the dependency tree and found the culprit: ChromaDB relies heavily on hnswlib
and stores its metadata in underlying SQLite/JSON formats. Because my local local_vector_db
was generated on Windows using Python 3.13, the specific versions of hnswlib
and ChromaDB serialized the metadata differently than the older Python 3.11 Linux packages running in the Docker container.
The cloud container was essentially trying to read a SQLite/JSON schema it didn't recognize, resulting in the missing _type key.
The Rabbit Hole (And When to Pivot)
My initial engineering instinct was to force parity by downgrading my local Windows packages to match the Docker container's versions. This was a mistake.
Attempting to downgrade hnswlib
and related C-dependent packages on Windows triggered a nightmare of missing C++ Build Tools errors. I spent an hour fighting the OS rather than solving the actual problem. A good engineer knows when to stop digging a hole and step back.
The Resolution: Containerize the Data Generation
I realized that if the application was going to run inside a Docker container, the data it consumed needed to be generated inside that exact same environment:
docker run -v ${PWD}:/app -it my-ai-agent-image python ingest.py
By running ingest.py inside the container, the local_vector_db
was generated using the exact Linux/Python 3.11 dependencies it would be read by. This guaranteed perfect schema parity. The KeyError
vanished, and the agent initialized flawlessly.