{"slug": "memorybox-ai-powered-semantic-search-for-your-memories", "title": "MemoryBox — AI-Powered Semantic Search for Your Memories", "summary": "A developer built MemoryBox, an open-weight AI pipeline that lets users search a personal photo archive using natural-language descriptions of moments rather than filenames or dates. The system uses Qwen3-VL-2B-Instruct to generate image descriptions, Qwen3-Embedding-0.6B to convert them into 1024-dimensional vectors, and PostgreSQL with pgvector for semantic search, with a Spring Boot backend, React frontend, and Docker configuration. The developer kept the model layer separate from the backend so the Apache 2.0-licensed checkpoints can be swapped or eventually run locally for greater privacy.", "body_md": "*This is my submission for the Hacktoberfest Weekend Challenge: Build for a Friend.*\n\nI built this for a friend who has a lot of photos and memories that are meaningful to them, but finding a particular one later can be surprisingly difficult.\n\nThey might remember the moment clearly:\n\n\"That evening when we went out.\"\n\n\"That photo with everyone.\"\n\n\"The day we were playing tennis.\"\n\nBut remembering a moment isn't the same as remembering a filename, folder, or date.\n\nThat was the problem I wanted to solve for them.\n\nSo I built **MemoryBox** — a personal memory archive where you can search for photos based on what you remember about the moment, rather than how the file is stored.\n\nInstead of asking:\n\n\"What was that file called?\"\n\nyou can ask:\n\n\"Show me the photos from our tennis day.\"\n\nMemoryBox uses open-weight AI to understand uploaded photos, turn that understanding into semantic representations, and make those memories searchable.\n\n🎥 **Demo video:** [https://youtu.be/TgyCizkhFz4](https://youtu.be/TgyCizkhFz4)\n\n🌐 **Live app:** [https://memorybox-frontend.onrender.com/](https://memorybox-frontend.onrender.com/)\n\nThe demo shows the complete flow: uploading a photo, having the AI understand it, and searching for the resulting memory using natural language.\n\n💻 **GitHub:** [https://github.com/asnamobin-hue/memorybox](https://github.com/asnamobin-hue/memorybox)\n\nThe repository contains the Spring Boot backend, React frontend, Docker configuration, and Python embedding helper.\n\nThe core of MemoryBox is an open-weight AI pipeline.\n\n```\nPhoto\n  │\n  ▼\nQwen3-VL-2B-Instruct\n  │\n  ▼\nAI-generated image description\n  │\n  ▼\nQwen3-Embedding-0.6B\n  │\n  ▼\n1024-dimensional embedding\n  │\n  ▼\nPostgreSQL + pgvector\n  │\n  ▼\nSemantic memory search\n```\n\nFor image understanding, I used **Qwen/Qwen3-VL-2B-Instruct**.\n\nFor embeddings, I used **Qwen/Qwen3-Embedding-0.6B**.\n\nBoth model checkpoints are published under the Apache 2.0 license.\n\nWhen a photo is uploaded, the vision model generates a description of what is happening in the image — including things like people, activities, objects, and surroundings.\n\nThat description is then passed to the embedding model, which converts it into a 1024-dimensional vector.\n\nThe vector is stored in **PostgreSQL with pgvector**.\n\nWhen someone searches for a memory, the search text is embedded and compared against the stored vectors to find semantically related memories.\n\nThe application is built with:\n\nThe deployed version currently calls the models through Hugging Face inference. The application keeps the model layer separate from the rest of the backend, so the models can be changed without redesigning the entire application.\n\nThat separation was important to me because I didn't want the AI part of the project to become permanently tied to one proprietary API.\n\nI chose open-weight models because I wanted control over the most important part of the application.\n\nMemoryBox isn't just using AI to generate a nice description.\n\nThe model output becomes part of the actual data pipeline:\n\n**photo → understanding → embedding → vector search**\n\nUsing separate open-weight models for those stages means I can inspect the models, change them, and experiment with different approaches without rebuilding the entire application around a single closed provider.\n\nIt also gives me a path toward a more private version of MemoryBox.\n\nThe current public deployment uses hosted inference because that made the project practical to deploy during the challenge. But the model checkpoints are available for local use, so a future version could move inference closer to the user's device or private server.\n\nFor a project dealing with personal memories, that possibility matters.\n\nA closed API can be convenient, but I don't want convenience to be the only option.\n\nOpen models gave me a different starting point: build the application around models that can be replaced, inspected, and eventually run in an environment I control.\n\nOne of the searches I tested was:\n\n**\"tennis\"**\n\nThe system was able to retrieve a memory from my uploaded photos that was associated with our tennis day.\n\nThe stored memory also contains the AI-generated description and its embedding, so the search isn't dependent on the original filename.\n\nThat was the part that made the idea feel real to me.\n\nI wasn't searching for a file called `IMG_1234.jpg`.\n\nI was searching for something I remembered.\n\nThe current search isn't perfect either. Some unrelated memories can still appear after the strongest matches.\n\nThat's one of the things I want to improve next rather than pretending semantic search is already perfect.\n\nGetting the first version working locally was not the hardest part.\n\nGetting it to work in production was.\n\nI ran into:\n\nAt one point, photos were uploading successfully but the AI processing was failing.\n\nI didn't know whether the problem was the vision model, embeddings, PostgreSQL, or deployment.\n\nSo I temporarily exposed the actual processing error through the application instead of continuing to guess.\n\nThat eventually led me to the real problem.\n\nThe embedding service was returning the vector in a nested form:\n\n```\n[[1024 values]]\n```\n\nwhile my Java code was expecting:\n\n```\n[1024 values]\n```\n\nThe fix was small: unwrap the nested response before validating the 1024 dimensions.\n\nFinding it was not small.\n\nThat experience taught me that getting something to work on localhost and getting it to work for an actual user are very different problems.\n\nMemoryBox currently stores the uploaded image, the user's description, the AI-generated description, and the embedding associated with the memory.\n\nThe deployed application uses Render for the backend and PostgreSQL for the database, while model inference currently goes through Hugging Face.\n\nSo the current version is **not yet a fully local/private memory vault**.\n\nI want to be clear about that because these are personal photos.\n\nThe open-weight model choice gives me a path toward local or self-hosted inference in a future version, but that is not something I want to claim the current deployment already provides.\n\nThis project started with a simple problem about finding photos.\n\nIt ended up teaching me much more about building an actual AI-backed application.\n\nI had to understand how:\n\nfit together.\n\nI also learned that using AI isn't just about calling a model.\n\nThe interesting engineering is everything around it — choosing where the model fits, processing its output, storing it, searching it, handling failures, and making the entire pipeline reliable.\n\nMost importantly, I learned that a small personal problem can lead to a surprisingly deep engineering project when you actually try to make it work end-to-end.\n\nMemoryBox is working end-to-end now, but I don't consider it finished.\n\nThe biggest things I'd improve are:\n\nFor this weekend, I kept the scope realistic.\n\nThe goal wasn't to build the biggest AI application I could.\n\nIt was to build something for one real person that could actually be used.\n\nThis project started with a friend and a simple problem: having a memory in mind but not having an easy way to find the photo again.\n\nThat became MemoryBox.\n\n**How do you find a memory again when you remember the moment, but not the file?**\n\nThat's the problem I wanted to solve for them.", "url": "https://wpnews.pro/news/memorybox-ai-powered-semantic-search-for-your-memories", "canonical_source": "https://dev.to/asnamobin-hue/memorybox-ai-powered-semantic-search-for-your-memories-1gna", "published_at": "2026-10-04 18:36:31+00:00", "updated_at": "2026-10-04 18:42:58.573036+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "natural-language-processing", "ai-tools", "developer-tools"], "entities": ["MemoryBox", "Qwen3-VL-2B-Instruct", "Qwen3-Embedding-0.6B", "PostgreSQL", "pgvector", "Hugging Face", "Spring Boot", "React"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/memorybox-ai-powered-semantic-search-for-your-memories", "markdown": "https://wpnews.pro/news/memorybox-ai-powered-semantic-search-for-your-memories.md", "text": "https://wpnews.pro/news/memorybox-ai-powered-semantic-search-for-your-memories.txt", "jsonld": "https://wpnews.pro/news/memorybox-ai-powered-semantic-search-for-your-memories.jsonld"}}