Last week, I lost more time to AI infrastructure than to the actual creative work.
The task sounded simple: generate images through Vertex AI for a content pipeline.
What actually happened:
invalid_grant
permission denied
This is the part nobody tells you about AI workflows:
most failures are not model failures. They’re credential, project, and policy failures.
Here’s how I finally debugged the whole thing and got the image route working again.
At first, the failures looked unrelated.
I saw three different classes of errors:
invalid_grant: account not found
403 Permission denied
429 RESOURCE_EXHAUSTED
That usually means one of two things:
In my case, it was the second one.
The first useful move was brutally simple:
find the one script that the team actually trusts.
For us, that was:
~/clawd/ops/production/scripts/generate_panels.py
That became the source of truth.
Not old snippets.
Not half-working notebooks.
Not memory.
Once I checked the actual script, I immediately found one hidden problem:
PROJECT = "old-project-id"
The pipeline was still hardcoded to an old project.
So even when I updated the credentials, the requests were still going to the wrong place.
That alone explained a lot.
We had two different routes mixed together:
That sounds harmless, but it creates terrible debugging conditions.
Because the failure modes are different:
If you mix them, you start solving the wrong problem.
For example, this looked like a model problem at first:
429 RESOURCE_EXHAUSTED
But it turned out to be just a burned free-tier key.
Meanwhile the paid route was failing for a completely different reason.
Lesson: treat free and paid as separate systems, even if they use the same model.
Once I had the new Vertex JSON, I didn’t start by generating an image.
I started by checking whether the credential could even mint a token.
That kind of test saves time because it tells you whether the problem is:
In Python, the logic is basically:
from google.oauth2 import service_account
from google.auth.transport.requests import Request
creds = service_account.Credentials.from_service_account_file(
"vertex_ai_key.json",
scopes=["https://www.googleapis.com/auth/cloud-platform"],
)
creds.refresh(Request())
print(creds.token[:40])
If this step fails, don’t touch your prompts.
Don’t touch the model.
Don’t touch the rendering code.
You don’t have an image problem yet.
You have an auth problem.
This one burned the most time.
I created a new service account, everything looked correct, and then Google Cloud refused to create a JSON key.
The error turned out to be caused by this policy:
iam.disableServiceAccountKeyCreation
That wasn’t obvious from the first screen.
The UI showed one policy as “not enforced,” while a legacy constraint was still active somewhere above it.
That kind of mismatch is why cloud debugging feels cursed.
The practical fix was not to keep fighting the same project.
The practical fix was to create a clean personal project without inherited org-policy baggage.
That ended up being faster than trying to untangle admin policy state.
The final working setup looked like this:
Only after that did I consider the route fixed.
Not when the key existed.
Not when the policy screen looked green.
Not when the script stopped crashing.
Only when this produced an actual file:
outputs/nanobanana_vertex_test.png
That was the only result that mattered.
When an AI image pipeline breaks, I now check things in this order:
That order is much faster than randomly changing keys and re-running prompts.
For us, the final fix was not “better prompting.”
It was:
That’s not glamorous.
But it’s the difference between a pipeline you trust and a pipeline that only works when you’re lucky.
A lot of AI tooling discourse is still obsessed with models.
But once you work with these systems in production, the real bottleneck is often much more boring:
identity, permissions, quotas, and project hygiene.
The model can be state of the art.
If your project graph is a mess, you still won’t ship.
This is also exactly the kind of workflow I want to turn into a Terminal Skill.
Not because a skill should hide the cloud setup behind magic, but because the debug order should not live only in someone’s memory.
A useful vertex-ai-image-pipeline
skill would give an agent a repeatable checklist:
That is the broader idea behind Terminal Skills: turn messy, real operational workflows into reusable agent skills.
I will probably translate this article into a proper Terminal Skills use case next, because this is the kind of boring production workflow agents need more than another prompt template.
If you’ve had to debug a broken AI pipeline recently, I’d genuinely love to hear what failed first for you.