An overview of an e-commerce image optimization method using Google's AI models and vector embeddings to iteratively generate and score product alt text.
I saw a really cool post written by Gianluca Fiorelli shared on LinkedIn today and I'd love to share how we approach image optimization for e-commerce and multi-modal AI search.
The above image is located on the official Google Fitbit Air page. We upload the image alongside the source URL and the original alt text.
<img data-component-name="media_area_image" jscontroller="Tbys9b" jsaction="rcuQ6b:npT2md;k0bule:jAmaef" class="WeYPde" src="https://lh3.googleusercontent.com/2B_ulFLMDz6jJXNnoktx_QgeEMEDAt-UfvbmEmTitaG4Y1eUt6qIGblB7CiHJHnKw0lraLHMu1rC1RyOi-aF-EBr31WtlNY52A=w1156-h924-rw-nu" alt="" width="1156" height="924" ="eager" decoding="auto" fetchpriority="auto">
Our optimizer uses Google's latest model to generate vector embeddings of multimedia inputs to provide our optimization algorithm with semantic similarity scoring.
Here is the full list of attempts:
| Step | Match | Δ | Caption |
|---|---|---|---|
| 0 | 59% | . | |
| 1 | 78% | +19% | Four colorful wristbands are displayed diagonally on a light background. |
| 2 | 82% | +4% | Four colored fabric wristbands, red, blue, gray, and light green, are arranged diagonally on a white background. |
| 3 | 81% | -2% | Four woven fabric wristbands, red, blue, gray, and light green, are arranged diagonally on a light white background. |
| 4 | 79% | -2% | Four woven fabric wristbands: red, blue, gray, and light green, diagonally on white. |
| 5 | 80% | +1% | Four colorful woven fabric wristbands, red, blue, gray, and light green, are arranged diagonally on a white background. |
| 6 | 81% | +1% | Four colored fabric wristbands in red, blue, gray, and light green, shown diagonally on white. |
| 7 | 83% | +2% | Four colorful fabric wristbands, red, blue, gray, and light green, are arranged diagonally on a light background. |
| 8 | 82% | -1% | Four colorful fabric wristbands, red, blue, gray, and light green, are arranged diagonally on a bright white background. |
| 9 | 83% | +1% | Four colorful fabric wristbands: red, blue, gray, and light green, are arranged diagonally on a light background. |
| 10 | 83% | 0% | Four fabric wristbands: red, blue, gray, and light green, arranged diagonally on a light background. |
After ten iterations the best matching alt text is:
Four colorful fabric wristbands: red, blue, gray, and light green, are arranged diagonally on a light background.
Image optimizer is small part of our AI Influence app, Auxy.