{"slug": "how-amazon-scores-product-pages-with-a-vision-model-ocr-and-catboost-94-accuracy", "title": "How Amazon Scores Product Pages With a Vision Model, OCR and CatBoost: 94% Accuracy", "summary": "Amazon Marketplace's content-quality system for A+ Content product pages reached 0.94 accuracy and 0.95 F1 on a 20 percent holdout of 5,478 expert-labeled pages, according to an AWS Machine Learning blog post published November 5, 2026, beating a 0.80 accuracy and 0.84 F1 baseline from simply averaging vision-language model scores. The system renders each full page as one snapshot, has a frozen VLM score it on a four-dimension rubric (Visual Storytelling, Feature Presentation, A+ Layout, Conversion Optimization, 0 to 10 each), adds four OCR readability signals, and feeds the eight features to a CatBoost classifier trained on majority-vote labels from one marketing expert and four annotators across 10 categories. AWS reports sellers raised their quality score by 1.4 points out of 10 after revisions, and says high-quality A+ Content lifts sales 8 to 20 percent, though the results have not been independently verified.", "body_md": "Amazon Marketplace now grades product-page content with a recipe other teams can copy: take one snapshot of the whole page, ask a frozen vision-language model to score it on a rubric, add OCR readability scores, and let a small CatBoost classifier decide. On 5,478 labeled pages, AWS says the fused model reached 0.94 accuracy and a 0.95 F1 score, against 0.80 and 0.84 for a simple average of the model's scores.\n\nThe details come from an [AWS Machine Learning blog post published November 5, 2026](https://aws.amazon.com/blogs/machine-learning/how-amazon-marketplace-evaluates-content-quality-with-amazon-sagemaker-ai/?ref=explainx). It is a vendor write-up of an internal system, with measured results, so we separate what it reports from what has been independently checked (nothing yet).\n\n## TL;DR: the method and the numbers\n\n| Item | Detail (as reported by AWS) | \n|---|---|\n| What is scored | A+ Content, the rich-media widget on Amazon product detail pages | \n| Why it matters | Per AWS, sellers using high-quality A+ Content raise sales 8 to 20 percent versus those without it | \n| Input | One rendered snapshot of the full content experience | \n| VLM rubric | Visual Storytelling, Feature Presentation, A+ Layout, Conversion Optimization; 0 to 10 each, plus confidence and rationale | \n| OCR signals | Text Density, Character Density, Spacing, Overall Readability | \n| Classifier | CatBoost on eight features, trained on expert labels | \n| Labels | 5,478 pages, 10 categories, one marketing expert and four annotators, majority vote | \n| Best result | 0.94 accuracy, 0.95 F1 on a 20 percent holdout | \n| Baseline | 0.80 accuracy, 0.84 F1 (averaged VLM scores plus a fixed threshold) | \n| Explanations | SHAP values, turned into recommendations by an LLM | \n| Seller effect | Quality score up 1.4 points of 10 after revisions | \n\n## The problem: pages are judged as a whole\n\nAWS argues that checking assets one at a time misses the point. A blender page can have a sharp hero image, grammatical bullets and a well-lit lifestyle photo, and still fail. The bullets may advertise a 64-ounce pitcher while the photo shows a single-serve model, or the lifestyle shot may sit above the hero image. Those problems appear only when the page is read as a whole, so Amazon renders the entire page as a snapshot.\n\nThis fits a wider pattern we have covered: using a model as a judge, then calibrating it against humans. Compare [Amazon Bedrock AgentCore evaluations with decision models](https://explainx.ai/blog/amazon-bedrock-agentcore-evaluations-decision-models-jev-strands-decider-screening-layer-guide-2026), where a small judge screens agent outputs.\n\n## Step one: freeze the VLM, learn the weights\n\nA pretrained VLM prompted with a rubric scores from general knowledge, which does not always match ecommerce standards. Fine-tuning would need GPUs and, with only a few thousand labels, risks overfitting. Amazon chose a different path: keep the VLM and the prompt fixed, use its structured outputs as features, and train a [CatBoost](https://catboost.ai/?ref=explainx) classifier to predict expert labels.\n\nThe rubric has four dimensions, each with explicit criteria and point-deduction rules. For each one the VLM returns a score from 0 to 10, a confidence value and a rationale. The scores and confidences become features. The rationales go to sellers.\n\nBecause only the classifier is retrained, updating the system when annotations grow or criteria change takes a short CPU-only job. That is the engineering insight worth copying.\n\n## The labels: how trustworthy are they?\n\nOne ecommerce marketing expert and four professional annotators independently rated 5,478 product pages from 10 categories as high or low quality, then took a majority vote. The classes were nearly balanced, with 49.6 percent high quality, so a model cannot score well by guessing the majority class. Each annotator's Cohen's kappa against the consensus was between 0.845 and 0.915, which AWS describes as strong agreement.\n\nThat is a useful reference point for anyone building an evaluation set: five raters, explicit criteria and a reliability statistic.\n\n## Step two: add OCR for what the VLM misses\n\nVLMs capture overall meaning and composition but are weaker on fine readability: crowded text, tight line spacing. They can also favor long, detailed descriptions even when shoppers want something scannable. So Amazon extracts text blocks and positions with an OCR model and computes four layout scores.\n\nThe scores depend on layout rather than wording, so two pages with identical text can score differently. The fused model has eight features: four from the VLM and four from OCR.\n\nThe AWS authors put the design choice this way: OCR \"isn't a replacement signal,\" because it complements the VLM's holistic judgment with fine-grained text-readability information. CatBoost also suits this setup because the labeled dataset is small and tabular. For context, [AWS describes](https://aws.amazon.com/blogs/machine-learning/how-amazon-marketplace-evaluates-content-quality-with-amazon-sagemaker-ai/?ref=explainx) A+ Content as the rich media widget on the Amazon product detail page, where brands combine text, images and video to communicate product and brand value.\n\n## The results, in numbers\n\nOn a 20 percent holdout of about 1,100 pages:\n\n| Configuration | Accuracy | F1 | \n|---|---|---|\n| Baseline: average VLM scores, fixed threshold | 0.80 | 0.84 | \n| OCR-only CatBoost | 0.82 | 0.83 | \n| VLM-only CatBoost | 0.89 | 0.90 | \n| Fused OCR-VLM CatBoost | 0.94 | 0.95 | \n\nAWS summarizes the fused model as 14 percentage points better in accuracy and 11 points better in F1 than the baseline. In a second test on a single category with 499 content projects, it swapped the VLM backbone. With a lighter VLM the baseline scored 0.74 accuracy and the fusion 0.90; with a more capable one the baseline scored 0.79 and the fusion 0.94. AWS reads this as the OCR gains holding across models.\n\nOne caution: the chart AWS shows spreads results across runs, and the headline figures are averages on Amazon's own labels. They have not been independently replicated.\n\n## Step three: explain with SHAP, then recommend\n\nA label alone does not tell a seller what to fix. Amazon uses [SHAP](https://github.com/shap/shap?ref=explainx), which assigns each feature a value showing how far it pushed a prediction toward high or low quality. Because VLM and OCR scores feed the same classifier, SHAP puts them on one scale.\n\nAcross the holdout set the most influential features were OCR Character Density (mean absolute SHAP 1.1), OCR Overall Readability (1.0) and VLM Visual Storytelling (1.0), followed by A+ Layout (0.8), Feature Presentation (0.6) and Conversion Optimization (0.4). Both sources matter.\n\nFor each low-quality page, the system ranks the eight features by SHAP value, then passes the most negative ones, the category, best practices and the VLM rationales to an LLM that writes prioritized actions. If A+ Layout has a SHAP value of minus 1.2 and Visual Storytelling minus 0.3, the layout fix comes first.\n\n## Did it help sellers and sales?\n\nAmazon says the average post-revision quality score rose by 1.4 points on a 10-point scale, on a statistically significant sample. It also reports a worldwide A/B experiment that adjusted visibility of low-quality A+ Content based on the model's decisions, which produced a \"statistically meaningful improvement\" in Ordered Product Sales.\n\nNeither the size of the sales effect nor the sample size is given. Treat the direction as reported and the magnitude as unknown.\n\n## How it runs on SageMaker AI\n\nAWS maps each stage to a managed service: Processing Jobs for rendering and SHAP, SageMaker AI Inference for the VLM and OCR (real-time, asynchronous or batch), JumpStart for pre-trained OCR models, Feature Store for features, Training Jobs and Model Registry for the classifier, and AIOps and Pipelines for drift monitoring and retraining. A final stage uses scores and SHAP recommendations to condition a diffusion model that generates improved image assets for sellers.\n\nIf you build on AWS, related guides we have written include [SageMaker SDK v3 for per-camera quality inspection models](https://explainx.ai/blog/sagemaker-sdk-v3-per-camera-models-edge-quality-inspection-neo-step-functions-2026), [SageMaker SOCI indexed images for faster training jobs](https://explainx.ai/blog/sagemaker-soci-indexed-images-lazy-loading-training-jobs-start-seconds-guide-2026) and [distilling a 4B shopping agent on SageMaker](https://explainx.ai/blog/aws-sagemaker-seqkd-4b-shopping-agent-matches-glm-5-3-teacher-0-541-vs-0-540-distillation-guide-2026).\n\n## How to copy the recipe\n\n1. Define a rubric with explicit criteria and deductions, and keep it stable.\n2. Render the unit you care about as one image, not as parts.\n3. Ask a strong VLM for scores, confidence and a rationale in structured form.\n4. Add cheap, deterministic signals, such as OCR layout measures, for what the VLM handles poorly.\n5. Collect a few thousand expert labels, with several raters and a reliability check.\n6. Train a small tree model on the features, and hold out 20 percent.\n7. Use SHAP to rank what to fix, and have a model turn the ranking into plain advice.\n8. Monitor drift, and retrain the classifier, not the VLM, when standards move.\n\n## Caveats and open questions\n\n- **Self-reported.** All numbers come from Amazon's post about its own system and labels.\n- **Context limits.** AWS says standards differ across categories, languages and marketplaces, and its annotations cannot fully represent every context.\n- **Seller impact.** A quality score and visibility changes affect sellers' sales, so how errors are handled matters. The post does not describe appeals.\n- **Agents are coming.** How shoppers' own AI agents read these pages is a separate question. See[Amazon's seller assistant and Claude plugin](https://explainx.ai/blog/amazon-seller-assistant-claude-plugin-24-7-agents-fact-check-2026) and[Paul Graham on agentic commerce](https://explainx.ai/blog/paul-graham-amazon-banning-agents-startup-opportunity-agentic-commerce-2026) .\n\nAWS says its next steps are more efficient VLM backbones and inference methods to run the system on a global catalog.\n\n## Related reading\n\n*Details are accurate as of November 5, 2026, based on the AWS Machine Learning blog post. Figures are as reported by AWS.*", "url": "https://wpnews.pro/news/how-amazon-scores-product-pages-with-a-vision-model-ocr-and-catboost-94-accuracy", "canonical_source": "https://explainx.ai/blog/amazon-marketplace-a-plus-content-quality-vlm-ocr-catboost-shap-sagemaker-94-percent-accuracy-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog", "published_at": "2026-11-05 00:00:00+00:00", "updated_at": "2026-11-05 20:24:05.481769+00:00", "lang": "en", "topics": ["machine-learning", "computer-vision", "ai-products", "ai-tools"], "entities": ["Amazon", "Amazon Marketplace", "AWS", "CatBoost", "Amazon SageMaker AI", "A+ Content"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-amazon-scores-product-pages-with-a-vision-model-ocr-and-catboost-94-accuracy", "markdown": "https://wpnews.pro/news/how-amazon-scores-product-pages-with-a-vision-model-ocr-and-catboost-94-accuracy.md", "text": "https://wpnews.pro/news/how-amazon-scores-product-pages-with-a-vision-model-ocr-and-catboost-94-accuracy.txt", "jsonld": "https://wpnews.pro/news/how-amazon-scores-product-pages-with-a-vision-model-ocr-and-catboost-94-accuracy.jsonld"}}