If you make ads for a small brand with an AI video model, the first complaint is rarely the motion or the light. It is the label. A jar comes back with letters that are almost right, a logo that has melted a little, a line of small print that is now a different line of small print. To the brand that is not their product any more. We build Zaqero AI, which makes short product videos for small businesses. This post is about how we try to keep the label theirs: what is in production, what we built and measured on a test shot, and what is still open.
The cheapest fix is upstream. An earlier version of our pipeline "cleaned up" a product photo with an image model before handing it to the video model, setting the product on a plain grey ground. That meant the label the video model saw was already the image model's copy of it.
Now the product photo reaches the video model as it is. We cut the product out of its background and set it on light grey with a margin, keeping every product pixel; when the background is not plain enough to cut, the photo goes in as it is, only turned upright. Nothing is regenerated or restyled on the way. This part is live.
It helps, and it does not solve the problem: the video model still generates every frame, and small print drifts.
After a take comes back, we ask a vision model one narrow question about one frame: where are the four corners of the label (and, for a round thing like a jar, the left and right edges of the jar at the label's height)? It answers in JSON, as shares of the frame.
Code then checks the answer before anything trusts it: four points, inside the picture, a real four-sided shape, and big enough, at least about 1.5% of the frame. If the label is not clearly there, the answer is "not found" and the shot stays exactly as it was generated. That rule runs through everything below: when unsure, change nothing.
The obvious way to follow a label is to key it by colour. On our test jar that failed on the label's own lettering: the key jumped up to 28 px between frames.
What works is following the picture itself. Our patch tracker takes the label region from one reference frame and, on every other frame, searches for the move and scale (and, for things that turn, the rotation) that best match it, by normalised correlation, refining in smaller and smaller steps down to a tenth of a pixel.
Two details mattered more than the search:
On the test shot, a slow push-in on a candle jar, the first version matched the reference at 0.993 on average. The final version, measured on the rendered video against the picture, sits at 0.04-0.05 px RMS of shake, 0.14 px at worst. A track that matches worse than 0.85 at its worst moment is thrown away and the shot left alone.
Our first attempts pasted a new label image over the old one. It looked like a sticker every time: too sharp, too clean, frozen while the real label underneath kept catching the light.
So we stopped painting labels. On each frame we flatten the label region out of the picture, fit a smooth curve through it to get the surface's own light (dark side and bright side), and treat what stands apart from that curve as print. Only the lettering's pixels are filled in, from that same frame's clean surface around them and with its own fine noise. The label's edges, its bow, its light and its grain stay the footage's. Then the new print, if the brand asked for one, goes on in the same place, at the same size and shape. (That is also our rule: a label is edited in place, as it is, never covered by a different one.)
To keep the print from looking pasted on, we measure the shot instead of judging it by eye:
While measuring the shake we found that every video had a stutter we had stopped noticing. The video model returns 24 frames a second; we rendered at 30, so every fifth frame was a repeat. A label tracked to a twentieth of a pixel still looks rough if the footage under it hitches. We now render at 24. This change is live. Plainly:
We will write again when it has run on real videos, with those numbers.
If you are working on the same problem, the two ideas we would keep are: match against one reference, never the previous frame; and lift the old print off each frame instead of laying anything over it.
*Zaqero AI makes short product videos for small businesses from their own photos: [https://www.zaqero.app](https://www.zaqero.app)*