{"slug": "built-a-cover-only-recommender-with-clip-embeddings-and-a", "title": "Built a cover-only recommender with CLIP embeddings and a", "summary": "A developer built a cover-only book recommender that fuses CLIP vector embeddings with GLiNER-extracted entities via Reciprocal Rank Fusion, queried against the Hardcover API, and uses a two-tower neural hybrid model with DPP diversification for collaborative filtering. The system, deployed on AWS, includes an async ingestion loop that expands the catalog from failed searches, and the author questions the saturation point of cover-only CLIP at 50k-100k items.", "body_md": "# Built a cover-only recommender with CLIP embeddings and a\n\nThe semantic search pipeline is a hybrid: CLIP vector search fused with GLiNER-extracted entities queried against the Hardcover API via Reciprocal Rank Fusion. GLiNER runs ONNX, which is the right call for latency, but the NER step feels like a band-aid for CLIP's known weakness on text-heavy covers. When the cover is just a title in a distinctive font, the visual embedding captures something useful. When it's an abstract pattern, the entity extraction carries the load. RRF merging them is standard practice, but I'd want to see the weight tuning — equal weighting assumes both signals have comparable precision, which they rarely do.\n\nThe collaborative filtering side uses a two-tower neural hybrid model with DPP diversification. Two-tower is the right architecture for this scale — the item tower stays fixed during serving, only the user tower updates per session. But training on explicit \"Dislike/Like/Love\" feedback with only a few thousand items and sparse user interactions? That's a cold-start nightmare. The offline retrain cadence (daily full retrain, 2-hour fine-tune) suggests the author knows the embedding drift problem. Eugene Yan's discovery system design post is the right reference here, though the 2-hour incremental update feels aggressive for a model that sees maybe dozens of new ratings per window.\n\n``` python\n# DPP diversification snippet from bic-learn\ndef diversify_recommendations(embeddings, k=10, lambda_param=0.5):\n    kernel = embeddings @ embeddings.T\n    selected = []\n    for _ in range(k):\n        scores = np.diag(kernel)\n        for idx in selected:\n            scores -= lambda_param * kernel[idx] ** 2\n        next_idx = np.argmax(scores)\n        selected.append(next_idx)\n    return selected\n```\n\nThe diversification logic above is a greedy MAP approximation — standard, but the lambda_param needs per-genre tuning. Fantasy covers cluster tightly; literary fiction spreads out. A fixed lambda will over-diversify one and under-diversify the other.\n\nAWS deployment details are sparse in the writeup, but the async ingestion loop (search → Hardcover API → vector DB) is the growth engine. Every failed search that triggers an API call expands the catalog. Clever flywheel if the Hardcover rate limits hold.\n\nBiggest question: at what catalog size does cover-only CLIP saturate? My guess is 50k-100k items before genre confusion dominates. Would love to see retrieval@k curves as the corpus grows.\n\n[Next Watermarked AI text still fools most readers in blind tests →](/en/news/7205/)\n\n[these AI tool field notes](https://tanyan888.com/), with plenty of directly applicable cases.\n\n## All Replies （0）\n\nNo replies yet — be the first!", "url": "https://wpnews.pro/news/built-a-cover-only-recommender-with-clip-embeddings-and-a", "canonical_source": "https://promptcube3.com/en/news/7210/", "published_at": "2026-08-21 21:10:11+00:00", "updated_at": "2026-08-21 21:12:43.592814+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence"], "entities": ["CLIP", "GLiNER", "Hardcover API", "Reciprocal Rank Fusion", "DPP", "AWS", "Eugene Yan"], "alternates": {"html": "https://wpnews.pro/news/built-a-cover-only-recommender-with-clip-embeddings-and-a", "markdown": "https://wpnews.pro/news/built-a-cover-only-recommender-with-clip-embeddings-and-a.md", "text": "https://wpnews.pro/news/built-a-cover-only-recommender-with-clip-embeddings-and-a.txt", "jsonld": "https://wpnews.pro/news/built-a-cover-only-recommender-with-clip-embeddings-and-a.jsonld"}}