If you have ever built an image-to-image (I2I) consumer feature, you already know the dirty secret of prompt engineering: what works in a Cherry-Picked Twitter thread rarely survives contact with real user uploads.
A few weeks ago, I started building an AI Cartoonify effect. The goal seemed straightforward—let users upload a selfie or pet photo and transform it into a stylized 3D/2D cartoon while keeping the person recognizable.
At first, I tried throwing the reference image and a carefully crafted style prompt into a single generation pass. It was a disaster.
Take a typical torture-test upload: a guy with a thick beard, wearing thin wire-rimmed glasses, standing under harsh side-lighting, holding up a peace sign.
When I ran this through a standard single-model I2I workflow, I kept hitting a brick wall:
The root cause? A single pass forces the model to figure out what is in the image and how to deform it artistically at the exact same time.
To ship this to production, I had to decouple semantic understanding from spatial rendering. Here is how I rebuilt the pipeline by pairing ChatGPT Image 2.5 with Nano Banana 2.1.
Instead of asking a single model to guess which pixels matter, I split the workload between two engines based on what they actually do best:
Rather than relying on static prompt templates, the backend builds a custom conditioning payload for every single upload. Here is a simplified look at the TypeScript structure driving the orchestration:
interface CartoonifyJob {
sourceImageUrl: string;
targetAesthetic: '3d-animation' | 'cel-shaded' | 'flat-illustration';
}
interface ConditioningPayload {
spatialReference: {
imageUrl: string;
edgeFidelity: 'strict' | 'balanced' | 'loose';
compositionLock: number; // 0.0 to 1.0
};
semanticTokens: {
identityLocks: string[]; // Extracted via ChatGPT Image 2.5
lightingCues: string[]; // Preserves mood without keeping raw photo noise
styleModifiers: string[]; // Drives the Nano Banana 2.1 shader
guardrails: string[]; // Anti-uncanny-valley negative tokens
};
}
async function buildCartoonPipeline(job: CartoonifyJob): Promise<ConditioningPayload> {
// Step 1: Extract fragile micro-features using ChatGPT Image 2.5
const analysis = await extractVisualAnchors(job.sourceImageUrl, {
model: 'chatgpt-image-2.5',
focusAreas: ['eyewear', 'facial_hair', 'hand_pose', 'garment_necklines']
});
// Step 2: Assemble the hybrid payload for Nano Banana 2.1
return {
spatialReference: {
imageUrl: job.sourceImageUrl,
edgeFidelity: 'balanced',
compositionLock: 0.70 // Enough room to exaggerate eyes/proportions
},
semanticTokens: {
identityLocks: analysis.detectedFeatures,
// e.g., ["thin gold wire-rimmed glasses", "trimmed full beard", "two-finger peace sign"]
lightingCues: [analysis.primaryLightingDirection, "soft rim light"],
styleModifiers: [
"stylized 3D character render",
"expressive stylized proportions",
"clean subsurface scattering"
],
guardrails: [
"photorealistic pores",
"hyper-detailed skin wrinkles",
"melted eyeglass frames",
"asymmetrical pupils"
]
}
};
}
Moving to a dual-model setup solved the macro architecture, but getting consistent outputs across thousands of edge cases took some serious trial and error.
Nano Banana 2.1 is fantastic at keeping macro composition intact—head tilt, shoulder angle, and background silhouettes stay rock solid when using a reference image. However, when you relax the reference lock enough to allow cartoon exaggeration (like slightly larger eyes or smoother jawlines), micro-details vanish. By having ChatGPT Image 2.5 explicitly inject tokens like "thin gold wire-rimmed glasses" into the prompt stream, Nano Banana 2.1 is forced to redraw those micro-details cleanly in the new cartoon style rather than blurring them away.
User photos usually have terrible lighting—fluorescent office bulbs, harsh bathroom shadows, or blown-out window backlighting. If you rely purely on image conditioning, the cartoon output inherits those muddy color casts. By extracting only the directional intent of the lighting (e.g., "warm side-lighting") via keywords and letting Nano Banana 2.1 re-render the actual color palette from scratch, the output looks like it came out of an animation studio rather than a dark bedroom.
The #1 reason AI cartoons look creepy is leftover high-frequency skin texture. When a model puts stylized cartoon eyes on a face that still has realistic pores and nasolabial folds, the human brain immediately rejects it. Hard-coding anti-realism guardrails (photorealistic pores, raw skin texture) into the Nano Banana 2.1 conditioning step was the single biggest quality unlock.
I have shipped this exact two-stage pipeline (ChatGPT Image 2.5 + Nano Banana 2.1) into production.
If you want to see how it handles tricky portraits, pets, or weird lighting—or if you just want to try breaking the semantic lock with your own photos—you can play with the live implementation here:
👉 Try the Live Tool: AI Cartoonify on VioEvo
As image models get faster and more specialized, chaining them together is becoming much more practical than waiting for one "god model" to do everything perfectly in a single shot. Using a heavy semantic model to write dynamic conditioning instructions for a fast spatial renderer has completely changed how I approach I2I features.
How are you all handling identity retention vs. style strength in your own generative pipelines? Drop a comment below if you've found other workarounds!